This commit prevents use-after-free of routes belonging to protocols
which have been already destroyed, delaying also all the protocols'
shutdown until all of their routes have been finally propagated through
all the pipes down to the appropriate exports.
The use-after-free was somehow hypothetic yet theoretically possible in
rare conditions, when one BGP protocol authors a lot of routes and the
user deletes that protocol by reconfiguring in the same time as next hop
update is requested, causing rte_better() to be called on a
not-yet-pruned network prefix while the owner protocol has been already
freed.
In parallel execution environments, this would happen an inter-thread
use-after-free, causing possible heisenbugs or other nasty problems.
This basically means that:
* there are some more levels of indirection and asynchronicity, mostly
in cleanup procedures, requiring correct lock ordering
* all the internal table operations (prune, next hop update) are done
without blocking the other parts of BIRD
* the protocols may get their own loops very soon
There is a simple universal IO loop, taking care of events, timers and
sockets. Primarily, one instance of a protocol should use exactly one IO
loop to do all its work, as is now done in BFD.
Contrary to previous versions, the loop is now launched and cleaned by
the nest/proto.c code, allowing for a protocol to just request its own
loop by setting the loop's lock order in config higher than the_bird.
It is not supported nor checked if any protocol changed the requested
lock order in reconfigure. No protocol should do it at all.
In some specific configurations, it was possible to send BIRD into an
infinite loop of recursive next hop resolution. This was caused by route
priority inversion.
To prevent priority inversions affecting other next hops, we simply
refuse to resolve any next hop if the best route for the matching prefix
is recursive or any other route with the same preference is recursive.
Next hop resolution doesn't change route priority, therefore it is
perfectly OK to resolve BGP next hops e.g. by an OSPF route, yet if the
same (or covering) prefix is also announced by iBGP, by retraction of
the OSPF route we would get a possible priority inversion.
* internal tables are now more standalone, having their own import and
export hooks
* route refresh/reload uses stale counter instead of stale flag,
allowing to drop walking the table at the beginning
* route modify (by BGP LLGR) is now done by a special refeed hook,
reimporting the modified routes directly without filters
Channels have now included rt_import_req and rt_export_req to hook into
the table instead of just one list node. This will (in future) allow for:
* channel import and export bound to different tables
* more efficient pipe code (dropping most of the channel code)
* conversion of 'show route' to a special kind of export
* temporary static routes from CLI
The import / export states are also updated to the new algorithms.
Routes are now allocated only when they are just to be inserted to the
table. Updating a route needs a locally allocated route structure.
Ownership of the attributes is also now not transfered from protocols to
tables and vice versa but just borrowed which should be easier to handle
in a multithreaded environment.
Pipes copy the original rte with old values, so they require rte to be
exported with stored tmpattrs. Other protocols access stored attributes
using eattr list, so they require rte to be exported with expanded
tmpattrs. This is temporary hack, we plan to remove whoe tmpattr mechanism.
Thanks to Paul Donohue for the bugreport.
In most cases of export there is no need to store back temporary
attributes to rte, as receivers (protocols) access eattr list anyway.
But pipe copies the original rte with old values, so we should store
tmpattrs also during export.
Thanks to Paul Donohue for the bugreport.
If there are roa_check() calls in channel filters, then the channel
subscribes to ROA table notifications, which are sent when ROA tables
are updated (subject to settle time) and trigger channel reload or
refeed.
The patch add support for per-channel debug flags, currently just
'states', 'routes', and 'filters'. Flag 'states' is used for channel
state changes, remaining two for routes passed through the channel.
The per-protocol debug flags 'routes'/'filters' still enable reporting
of routes for all channels, to keep existing behavior.
The patch causes minor changes in some log messages.
Logging as a result of triggered receive limit in import table code
accesset rte->net, which was not filed yet.
Thanks to Pier Carlo Chiodi for the bugreport.
Put new non-best routes to the end of list instead of the second
position. Put updated routes to their old position. Position is changed
just by best route selection.
The old code stored route verdicts and temporary routes directly in
rtable. The new code do not store received routes (it immediately
compares them with exported routes and resolves conflicts) and uses
internal bitmap to keep track of which routes were received and which
needs to be reinstalled.
By not putting 'invalid' temporary routes to rtable, we keep rtable
in consistent state, therefore scan no longer needs to be atomic
operation and could be splitted to multiple events.
Use a hierarchical bitmap in a routing table to assign ids to routes, and
then use bitmaps (indexed by route id) in channels to keep track whether
routes were exported. This avoids unreliable and inefficient re-evaluation
of filters for old routes in order to determine whether they were exported.
Underlying (IGP) route may lead to PtP link, in this case it does not
need gateway. Which is different than direct route without gateway.
When recursive (BGP) route uses PtP route, it should not use recursive
next hop as immediate next hop, while for direct routes it should.
The patch implements optional internal export table to a channel and
hooks it to BGP so it can be used as Adj-RIB-Out. When enabled, all
exported (post-filtered) routes are stored there. An export table can be
examined using e.g. 'show route export table bgp1.ipv4'.
The temporary atttributes are no longer removed by ea_do_prune(), but
they are undefined by store_tmp_attrs() protocol hooks. This fixes
several bugs where temporary attributes were removed when they should
not or not removed when they should be. The flag EAF_TEMP is no longer
needed and was removed.
Update all protocol make_tmp_attrs() / store_tmp_attrs() hooks to use
helper functions and to handle unset attributes properly.
Also fix some related bugs like improper handling of empty eattr list.
Keep track of whether OSPF tmpattrs are actually defined for given route
(using flags in rte->pflags). That makes them behave more like real
eattrs so a protocol can define just a subset of them or they can be
undefined by filters.
Do not set ospf_metric2 for other than type 2 external OSPF routes and do
not set ospf_tag for non-external OSPF routes. That also fixes a bug
where internal/inter-area route propagated from one OSPF instance to
another is initiated with infinity ospf_metric2.
Thanks to Yaroslav Dronskii for the bugreport.
Route flags are mosty internal state of rtable, they are not significant
to whether a route has changed. With the old code, all routes received as
a part of enhanced route refresh are always re-announced to other peers
due to change in REF_STALE.
One of previous workarounds for phantom route avoidance breaks export
counters by expanding sending of spurious withdraws, which are send when
we are not sure whether we have advertised that routes in the past.
If not, then export counter is decreased, but it was not increased
before, so it overflows under zero.
The patch fixes that by sendung spurious withdraws, but not counting them
on export counter. That may lead to error in the other direction, but that
happens only as a race condition (i.e., in normal operation filters
return proper values about old route export state).
The earlier fix loosen conditions for not running filters on old
route when deciding about route propagation to a protocol to avoid
issues with ghost routes in some race conditions.
Unfortunately, the fix also caused back-propagation of withdraws. For
regular updates, back-propagation is prevented in import_control hooks,
but these are not called on withdraws. For them, import_control hooks
are called on old routes instead, changing (old, NULL) notification
to (NULL, NULL), which is ignored. By not calling export processing
in some cases, the withdraw is not ignored and is back-propagated.
This patch fixes that by contract conditions so the earlier fix is not
applied to back-propagated updates.
The patch implements optional internal import table to a channel and
hooks it to BGP so it can be used as Adj-RIB-In. When enabled, all
received (pre-filtered) routes are stored there and import filters can
be re-evaluated without explicit route refresh. An import table can be
examined using e.g. 'show route import table bgp1.ipv4'.
Once upon a time, far far away, there were the old Bird developers
discussing what direction of route flow shall be called import and
export. They decided to say "import to protocol" and "export to table"
when speaking about a protocol. When speaking about a table, they
spoke about "importing to table" and "exporting to protocol".
The latter terminology was adopted in configuration, then also the
bird CLI in commit ea2ae6dd0 started to use it (in year 2009). Now
it's 2018 and the terminology is the latter. Import is from protocol to
table, export is from table to protocol. Anyway, there was still an
import_control hook which executed right before route export.
One thing is funny. There are two commits in April 1999 with just two
minutes between them. The older announces the final settlement
on config terminology, the newer uses the other definition. Let's see
their commit messages as the git-log tool shows them (the newer first):
commit 9e0e485e50
Author: Martin Mares <mj@ucw.cz>
Date: Mon Apr 5 20:17:59 1999 +0000
Added some new protocol hooks (look at the comments for better explanation):
make_tmp_attrs Convert inline attributes to ea_list
store_tmp_attrs Convert ea_list to inline attributes
import_control Pre-import decisions
commit 5056c559c4
Author: Martin Mares <mj@ucw.cz>
Date: Mon Apr 5 20:15:31 1999 +0000
Changed syntax of attaching filters to protocols to hopefully the final
version:
EXPORT <filter-spec> for outbound routes (i.e., those announced
by BIRD to the rest of the world).
IMPORT <filter-spec> for inbound routes (i.e., those imported
by BIRD from the rest of the world).
where <filter-spec> is one of:
ALL pass all routes
NONE drop all routes
FILTER <name> use named filter
FILTER { <filter> } use explicitly defined filter
For all protocols, the default is IMPORT ALL, EXPORT NONE. This includes
the kernel protocol, so that you need to add EXPORT ALL to get the previous
configuration of kernel syncer (as usually, see doc/bird.conf.example for
a bird.conf example :)).
Let's say RIP to this almost 19-years-old inconsistency. For now, if you
import a route, it is always from protocol to table. If you export a
route, it is always from table to protocol.
And they lived happily ever after.
The new MRT protocol is responsible for periodic RIB table dumps in the
MRT format (RFC 6396). Also the existing code for BGP4MP MRT dumps is
refactored and splitted between BGP to MRT protocols, will be more
integrated into MRT in the future.
Example:
protocol mrt {
table "*";
filename "%N_%F_%T.mrt";
period 60;
}
It is partially based on the old MRT code from Pavel Tvrdik.
If export filter is changed during reconfiguration and a route disappears
between reconfiguration and refeed (e.g., if the route is a static route
also removed during the reconfiguration), the route is not withdrawn.
The issue was fixed for regular channels by an earlier patch. This patch
fixes the issue for channels in RA_ACCEPTED mode (first-pass-the-filter),
used by BGP with 'secondary' option.
If export filter is changed during reconfiguration and a route disappears
between reconfiguration and refeed (e.g., if the route is a static route
also removed during the reconfiguration), the route is not withdrawn.
The patch fixes that by adding tx reconfiguration timestamp.
This is a fundamental change of an original (1999) concept of route
processing inside BIRD. During import/export, there was a temporary
ea_list created which was to be used instead of the another one inside
the route itself.
This led to some confusion, quirks, and strange filter code that handled
extended route attributes. Dropping it now.
The protocol interface has changed in an uniform way -- the
`struct ea_list *attrs` argument has been removed from store_tmp_attrs(),
import_control(), rt_notify() and get_route_info().
This patch adds support for source-specific IPv6 routes to BIRD core.
This is based on Dean Luga's original patch, with the review comments
addressed. SADR support is added to network address parsing in confbase.Y
and to the kernel protocol on Linux.
Currently there is no way to mix source-specific and non-source-specific
routes (i.e., SADR tables cannot be connected to non-SADR tables).
Thanks to Toke Hoiland-Jorgensen for the original patch.
Minor changes by Ondrej Santiago Zajicek.
A filter should log messages only if executed explicitly (e.g., during
route export or route import). When a filter is executed for technical
reasons (e.g., to establish whether a route was exported before), it
should run silently.
Add proper support for per-nexthop onlink flag in routes to handle next
hop addresses that are not covered by interface IP ranges. Supported by
kernel and static protocols.
Thanks to Vincent Bernat for the idea.
Some code cleanup, multiple bugfixes, allows to specify also channel
for 'show route export'. Interesting how such apparenty simple thing
like show route cmd has plenty of ugly corner cases.
Basic support for SAFI 4 and 128 (MPLS labeled IP and VPN) for IPv4 and
IPv6. Should work for route reflector, but does not properly handle
originating routes with next hop self.
Based on patches from Jan Matejka.
Anyway, Bird is now capable to insert both MPLS routes and MPLS encap
routes into kernel.
It was (among others) needed to define platform-specific AF_MPLS to 28
as this constant has been assigned in the linux kernel.
No support for BSD now, it may be added in the future.
Dropped struct mpnh and mpnh_*()
Now struct nexthop exists, nexthop_*(), and also included struct nexthop
into struct rta.
Also converted RTD_DEVICE and RTD_ROUTER to RTD_UNICAST. If it is needed
to distinguish between these two cases, RTD_DEVICE is equivalent to
IPA_ZERO(a->nh.gw), RTD_ROUTER is then IPA_NONZERO(a->nh.gw).
From now on, we also explicitely want C99 compatible compiler. We assume
that this 20-year norm should be known almost everywhere.
Kernel protocol calls rt_export_merged(), which used @rte_update_pool for
temporary allocations, supposing it is called from other functions from
rt-table.c that handles locking and flushing of the linpool. Therefore,
linpool was not flushed properly and memory leaked.
Add linpool argument to rt_export_merged() and use @krt_filter_lp when
called from kernel protocol.
Thanks to Justin Cattle and Alexander Frolkin for the bugreport.
(Commit squashed and updated by Ondrej Zajicek)
Many protocols do almost the same when creating a rte_update request
before calling rte_update2(). This commit should simplify the protocol
side of the route-creation routine.
The patch adds support for channels, structures connecting protocols and
tables and handling most interactions between them. The documentation is
missing yet.
Explicit setting of AF_INET(6|) in IP socket creation. BFD set to listen
on v6, without setting the V6ONLY flag to catch both v4 and v6 traffic.
Squashing and minor changes by Ondrej Santiago Zajicek
Symbol lookup by cf_find_symbol() not only did the lookup but also added
new void symbols allocated from cfg_mem linpool, which gets broken when
lookups are done outside of config parsing, which may lead to crashes
during reconfiguration.
The patch separates lookup-only cf_find_symbol() and config-modifying
cf_get_symbol(), while the later is called only during parsing. Also
new_config and cfg_mem global variables are NULLed outside of parsing.
New data types net_addr and variants (in lib/net.h) describing
network addresses (prefix/pxlen). Modifications of FIB structures
to handle these data types and changing everything to use these
data types instead of prefix/pxlen pairs where possible.
The commit is WiP, some protocols are not yet updated (BGP, Kernel),
and the code contains some temporary scaffolding.
Comments are welcome.
The new RIP implementation fixes plenty of old bugs and also adds support
for many new features: ECMP support, link state support, BFD support,
configurable split horizon and more. Most options are now per-interface.
In some circumstances during reconfiguration, routes propagated by pipes
to other tables may hang there even after the primary routes are removed.
There is already a workaround for this issue in the code which removes
these stale routes by flush process when source protocols are shut down.
This patch is a cleaner fix and allows to simplify the flush process
When route was propagated to another rtable through a pipe and then the
pipe was reconfigured softly in such a way that any subsequent route
updates are filtered, then the source protocol shutdown didn't clean up
the route in the second rtable which caused stale routes and potential
crashes.
Temporary dummy routes created by a kernel protocol during routing table
scan get mixed with real routes propagated from another kernel protocol
through a pipe.
The RAdv protocol could be configured to change its behavior based on
availability of routes, e.g., do not announce router lifetime when a
default route is not available.
When 'import keep rejected' protocol option is activated, routes
rejected by the import filter are kept in the routing table, but they
are hidden and not propagated to other protocols. It is possible to
examine them using 'show route rejected'.
Allows to send and receive multiple routes for one network by one BGP
session. Also contains necessary core changes to support this (routing
tables accepting several routes for one network from one protocol).
It needs some more cleanup before merging to the master branch.
When a protocol went down, all its routes were flushed in one step, that
may block BIRD for too much time. The patch fixes that by limiting
maximum number of routes flushed in one step.
The nest-protocol interaction is changed to better handle multitable
protocols. Multitable protocols now declare that by 'multitable' field,
which tells nest that a protocol handles things related to proto-rtable
interaction (table locking, announce hook adding, reconfiguration of
filters) itself.
Filters and stats are moved to announce hooks, a protocol could have
different filters and stats to different tables.
The patch is based on one from Alexander V. Chernikov, thanks.
Hostcache is a structure for monitoring changes in a routing table that
is used for routes with dynamic/recursive next hops. This is needed for
proper iBGP next hop handling.
In usual configuration, such export is already restricted
with the aid of the direct protocol but there are some
races that can circumvent it. This makes it harder to
break kernel device routes. Also adds an option to
disable this restriction.
- BSD kernel syncer is now self-conscious and can learn alien routes
- important bugfix in BSD kernel syncer (crash after protocol restart)
- many minor changes and bugfixes in kernel syncers and neighbor cache
- direct protocol does not generate host and link local routes
- min_scope check is removed, all routes have SCOPE_UNIVERSE by default
- also fixes some remaining compiler warnings
It seems that by adding one pipe-specific exception to route
announcement code and by adding one argument to rt_notify() callback i
could completely eliminate the need for the phantom protocol instance
and therefore make the code more straightforward. It will also fix some
minor bugs (like ignoring debug flag changes from the command line).
When uncofiguring the pipe and the peer table, the peer table was
unlocked when pipe protocol state changed to down/flushing and not to
down/hungry. This leads to the removal of the peer table before
the routes from the pipe were flushed.
The fix leads to adding some pipe-specific hacks to the nest,
but this seems inevitable.
If protocol announces a route, route is accepted by import filter to
routing table, and later it announces replacement of that route that is
rejected by import filter, old route remains in routing table.
Routes comming through pipe from primary to secondary table were
filtered by both EXPORT and IMPORT filters, but they should be
only filtered by EXPORT filters.
contains all attributes, not just the temporary ones. This avoids having
to merge the lists inside protocols or doing searches on both of them.
Also, do filtering of routes properly. (I'd like to avoid it, but it's
needed at least in the krt protocol.)
show the routing table as exported to the protocol given resp. as returned
from its import control hook.
To get handling of filtered extended attributes right (even in the old
`show route where <filter>' command), the get_route_info hook gets an
attribute list and all protocol specific rte attributes are contained
there as temporary ones. Updated RIP to do that.
Added ea_append() which joins two ea_list's.
Please try compiling your code with --enable-warnings to see them. (The
unused parameter warnings are usually bogus, the unused variable ones
are very useful, but gcc is unable to control them separately.)
of calling the protocols manually.
Implemented printing of dynamic attributes in `show route all'.
Each protocol can now register its own attribute class (protocol->attr_class,
set to EAP_xxx) and also a callback for naming and formatting of attributes.
The callback can return one of the following results:
GA_UNKNOWN Attribute not recognized.
GA_NAME Attribute name recognized and put to the buffer,
generic code should format the value.
GA_FULL Both attribute name and value put to the buffer.
Please update protocols generating dynamic attributes to provide
the attr_class and formatting hook.
The changes are just too extensive for lazy me to list them
there, but see the comment at the top of sysdep/unix/krt.c.
The code got a bit more ifdeffy than I'd like, though.
Also fixed a bunch of FIXME's and added a couple of others. :)
definitely gone. Both rte_update() and rte_discard() have an additional
argument telling which table should they modify.
Also, rte_update() no longer walks the whole protocol list -- each table
has a list of all protocols connected to this table and having the
rt_notify hook set. Each protocol can also freely decide (by calling
proto_add_announce_hook) to connect to any other table, but it will
be probably used only by the table-to-table protocol.
The default debugging dumps now include all routing tables and also
all their connections.
o Introduced rte_cow() which should be used for copying on write the
rte's in filters. Each rte now carries a flag saying whether it's
a real route (possessing table linkage and other insignia) or a local
copy. This function can be expected to be fast since its fast-path
is inlined.
o Introduced rte_update_pool which is a linear memory pool used for
all temporary data during rte_update. You should not reference it directly
-- instead use a pool pointer passed to all related functions.
o Split rte_update to three functions:
rte_update The front end: handles all checking, inbound
filtering and calls rte_recalculate() for the
final version of the route.
rte_recalculate Update the table according to already filtered route.
rte_announce Announce routing table changes to all protocols,
passing them through export filters and so on.
The interface has _not_ changed -- still call rte_update() and it will
do the rest for you automagically.
o Use new filtering semantics to be explained in a separate mail.
o Now compatible with filtering.
o Learning of kernel routes supported only on CONFIG_SELF_CONSCIOUS
systems (on the others it's impossible to get it semantically correct).
o Learning now stores all of its routes in a separate fib and selects
the ones the kernel really uses for forwarding packets.
o Better treatment of CONFIG_AUTO_ROUTES ports.
o Lots of internal changes.
nodes having no routes attached. Such cleanup must be done from event handler
since most functions manipulating the routing tables expect network entries
won't disappear from under their hands and it's also probably faster when
done asynchronously.
o rte can now contain a pointer to both cached and uncached rta. Protocols
which don't need their own attribute caching can now just fill-in a rta,
link it to rte without any calls to attribute cache and call rte_update()
which will replace rte->attrs by a cached copy.
o In order to support this, one of previously pad bytes in struct rta
now holds new attribute flags (RTAF_CACHED). If you call rte_update()
with uncached rta, you _must_ clear these flags. In other cases rta_lookup()
sets it appropriately.
o Added rte_free() which is useful when you construct a rte and then the
circumstances change and you decide not to use it for an update. (Needed
for temporary rte's in kernel syncer...)
protocol callbacks for route insertion and deletion from the central table.
RIP should maintain its own per-protocol queue of existing routes, scan it
periodically and call rte_discard() for routes that have timed out.
loop detection. This is needed since both RIP and OSPF handle multiple
neighbors and they need to redistribute routes learned from each neighbor
to the remaining ones.
protocols and don't send route/interface updates to them and when they come up,
we resend the whole route/interface tables privately.
Removed the "scan interface list after protocol start" work-around.
