RMHAM RouterOS 7 Progress Report
Willem A. Schreüder AC0KQ
8/1/2022: RouterOS 7.10.2
In preparation for the eventual migration of the entire network to RouterOS 7, I
have been experimenting with RouterOS releases.
In the previous progress report I had concluded that RouterOS 7
was not yet ready for prime time. The current review suggests that RouterOS 7
has matured to the point that we can recommend its adoption. Some care is
needed in upgrading, so remote upgrades are generally not advisable. However,
in general the upgrade option is safe and as long as you have
RoMON access to the device, you can fix the routing issues that may
develop.
For these experiments I initially upgraded the RB750 and RB951 five port routers
on the NetLab. This generally went well, although older routers with less
memory seem to be slow in updating. No netboot was needed but in one
a power cycle was required.
I then proceeded to update the router at my QTH, my office and Saddleback.
These are all CCR1009 routers and the upgrade proceeded about as fast as a
normal update of RouterOS.
I also experimented with upgrading some VPN attached devices such as the RB2011
in my trailer, that hAP Lite at KD0NFS and a mAP I use for remote access. These
all proceeded smoothly except for operator induced mishaps.
Differences between an Upgrade and a Fresh Install
Upgrading the router is the path of least resistance, and is fairly clean. I
therefore recommend that you upgrade your RouterOS 6 instance and not do a fresh
install of RouterOS 7 unless it is a new device.
If instead of upgrading you reset the configuration and use an export to
selectively restore the configuration, a number of things will break due to
differences between RouterOS 6 and RouterOS 7, so you are better off just
upgrading and letting RouterOS deal with these differences.
Of course, as we add new devices to the network and start with a fresh RouterOS
7 configuration, or if you are simply an intrepid soul like me and want to do
the upgrade yourself to better understand what is going on, you will notice that
there are a few differences between a freshly created configuration and an
upgraded one.
- The name of an upgraded version 2 OSPF Instance is default-v2,
while a newly created instance will be named ospf-instance-1.
- The name of an upgraded OSPF Area is backbone-v2, while a
newly created area will be ospf-area-1
- The OSPF Router ID for an upgraded OSPF Instance will be the IP
address of the bridge, while a newly created instance will be named main
by default and the Router ID is dynamically assigned.
The name of the OSPF Instance and OSPF Area is not critical, and a
router with a different name can exchange data with another router where the
instance and area have different names without any difficulty. However, for now
I would recommend that you use the same names on a fresh install that the
upgrade script will use for consistency's sake. You can change these names at
any time and RouterOS will use that new name consistently, so you do not have to
go and change it everywhere it is used.
The Router ID is of course critical as it is used in exchanges with other
routers. In a new instance, RouterOS selects an IP from a Virtual Routing and
Forwarding (VRF) interface, which will likely be the IP address of the bridge.
Since this interface will not drop when a cable is unplugged, it is a good
choice. However, for sanity's sake, it is best to just explicitly set the
Router ID for the OSPF Instance.
I will do a complete HOWTO on setting up a new RouterOS 7 box Some Time
Soon™.
Upgrading from RouterOS 6 to RouterOS 7
Upgrading the router from RouterOS 6 to RouterOS 7 is clean and relatively
painless. To perform the upgrade you may first have to upgrade to the latest
stable or long term version of RouterOS 6. Then in System
Packages do Check For Updates and select the upgrade channel.
Click Download&Install which should take you to the latest stable
version of RouterOS 7 which was 7.10.2 at the time of writing. Upgrading the
RouterBOARD firmware requires a second reboot.
If you do not use OSPF or Recursive Routing, upgrading to RouterOS 7 does not
require any further action. Static routes, VPNs and the like should work as
before. The only known problem reported by James KI0KN is that on the CCR2004
router which uses the ARM64 architecture, VPNs die after a few minutes. The
cause of this is not known at this time but my guess is that it is a bug in the
64 bit implementation of RouterOS. No other routers have been reported to have
this issue.
If you use OSPF and/or Recursive Routes, your adventure continues.
The following problems occur during upgrading:
- OSPF Interface Templates are created only for physical ports, not
VPNs. If you have VPN links over which you run OSPF, you have to manually
re-create them. The VPNs will reconnect, but make sure you can get to the
router directly or by RoMON so that you can fix OSPF.
- The OSPF filters should upgrade but will be incorrect. Specifically
the default route will not be filtered which could cause problems with VPNs
flapping.
- Recursive Routes do not upgrade correctly, or at least not if you set
it up using the RMHAM recommended procedure for RouterOS 6. You will need to
manually fix the Scope and Target Scope settings.
Fixing OSPF
First, add a new filter by selecting Routing Filters and then clicking
the blue + to add a new rule
The rule is somewhat long to enter so it may best be done from the command line
/routing filter rule add chain=ospf-rmham disabled=no rule="if (dst in 10.0.0.0/8 || dst in 172.16.0.0/16 || dst in 192.168.0.0/16 && not dst in 192.168.0.0/22) {accept} else {reject}"
This rule selects only the specific subnets we want to route, i.e. 10.0.0.0/8,
172.16.0.0/16 and 192.168.0.0/16 excluding 192.168.[0-3].0/24. This could be
written as a sequence of rules, but doing it as a single rule makes managing it
using DevDB easier.
Next we update the OSPF Instance by clicking Routing OSPF,
selecting the Instances tab and double clicking the default-v2
instance which is what the OSPFv2 instance would be called.
Make the OSPF Instance look like the above image, except that the Router ID should be the address of the bridge on your router.
- The Version, VRF and Router ID should already be
correct.
- For Redistribute the upgrade script will check dhcp and
modem. This probably doesn't hurt but I turned it off to be consistent
with fresh installs, so that all that
remains is connected, static and vpn.
- Set both the Out Filter and the In Filter to ospf-rmham.
Click Apply or OK to save.
You can now return to the Routing Filters and remove any old
ospf-in and ospf-out rules since they are no longer useful.
Note that in RouterOS 7, you do not need to tell OSPF that you want it to
distribute specific subnets. (In an earlier writeup, I had incorrectly
suggested that you do that with Area Ranges but this is wrong so do not
do that.) All the routing types you indicate on the OSPF Instance will be
distributed unless the Routing Filters prohibit it. Using the
ospf-rmham filter for both the In Filter and Out Filter
makes sure that the router distributes and accepts only the routes we select. It
is therefore no longer necessary to add a specific filter to hide your WAN.
Fixing OSPF Interface Templates
An important difference between RouterOS 6 and RouterOS 7 is that in RouterOS 7,
an OSPF neighbor relationship is not established across a link unless you
explicitly allow it using an OSPF Interface Template.
When upgrading, RouterOS will create templates for adjacent neighbors connected
via a microwave link, but not for VPN connections. The templates are very
similar. To add a template for the VPN, simply click the blue +
on the OSPF Interface Templates tab to create a new template. For the
existing templates, double-click on it.
The critical settings that must be present are:
- Interfaces selects the interfaces from the dropdown. For an incoming
VPN connection you need to first create a static interface as in RouterOS 6 by
copying the dynamically created interface in Interfaces.
- The template created by RouterOS during the upgrade will set both the
Interfaces and Networks to the subnet connected to the interface.
When setting the Interfaces I recommend that you remove the
Networks.
- Network Type will typically be ptp for point-to-point
connections, but may need to be broadcast or ptmp for links with
multiple routers.
- Cost should be set to 10 for microwave links and a high value for
VPNs. The high VPN value prevents traffic being routed via the VPN unless it is
a last resort. I have been using 5000 for the preferred VPN and 5500 for the
backup VPN.
- Priority is used to elect the designated router if the Network
Type is broadcast. The default is 128 if this is a new interface.
When upgrading from RouterOS 6, it defaults to 1 (lowest priority). Set it to
255 if you want it to always be the designated router, or 0 if you want it to
never be the designated router. For point-to-point connections the priority is
ignored.
I think the defaults work for pretty much everything else. Check back often
since I may change my mind.
OSPF Interface Templates by Subnet
At a site like Thorodin where there are many VPN connections, it is a chore set
up an Interface Template for every VPN connection. RouterOS 7 provides
an alternative that can be applied to all connections on a subnet.
As shown in the above image, instead of selecting specific interfaces, you can
set the OSPF link properties for all interfaces in that subnet. Note that in
this instance the Interfaces setting is blank and the Networks is
set to the VPN subnet. This sets the Network Type and Cost for
all the VPN connections.
A huge advantage of this approach is that is obviates the need to create a static
interfaces for each VPN. If for some VPN connection you want to use a different
weight, you can create another OSPF Interface Template that explicitly
references that connection, which must, of course, be static. It appears that
when the Interfaces is set, that overrides the network setting.
The same technique can also be used to assign the weight for all ethernet
interfaces using, for example, the 10.20.0.0/16 subnet. However, my inclination
is to use this technique only for VPNs and retain the ability to steer traffic
by link by having an interface for each ethernet connection.
Fixing Recursive Routes
In a Recursive Route, we use a target beyond the router to decide whether to
send traffic to that gateway router. This helps to ensure connectivity beyond
the gateway router exists.
In RouterOS 7 the concepts in Recursive Routes remain the same as in RouterOS 6,
but the implementation is slightly different. The upgrade script will cause
this default route to be unreachable if it was set using the previous
recommendation. So you need to update this manually as shown below:
- Set a route via the relevant gateway to a reliable remote target. In the
above we use 1.1.1.1 as the remote target and the gateway
165.140.185.113.
- The critical setting is that the Scope should be a low value. Use 10
unless you know what you are doing. This will automatically set the Target
Scope to 10 also. The Distance should be 1.
- Now set up the default route 0.0.0.0/0 using as 1.1.1.1 as the
gateway.
- Set Check Gateway to ping. This will ping 1.1.1.1 every 10
seconds to make sure it is reachable.
- The critical setting is that the Target Scope should be a greater
than the of the first route. Here I used 11. The default Scope
of the default route is not important and will default to 30.
The Distance should be set that is it smaller than the alternative
default route, so 2 is a good value.
Since we now filter by inclusion rather than exclusion, it is no longer
necessary to add a routing filter to cull the routes introduced by recursive
routing.
Miscellaneous Observations
Here are a few things I found that doesn't fit anywhere else.
- RouterOS 7 requires that in a PPP secret you must set the Local
Address. In RouterOS 6 that was optional.
- You can see what routes are being filtered in IP Routes by selecting
Filtered is yes. This is new in RouterOS 7.
- In IP Routes the + in a route type DAo+ means that
there are multiple Dynamic Active OSPF routes of equal weight, a.k.a as Equal
Cost Multi-Path (ECMP).
- In a few instances I found that when a VPN dropped and reconnected, it
would reconnect as a dynamic interface even when a static interface was defined.
I do not know whether this was caused by me messing around with the configuration or
whether this is a problem in RouterOS 7.
- In earlier versions of RouterOS 7 it was necessary to set the Routing
Table on the OSPF instance to main in order for routing to work.
This no longer seems to be necessary or I may just have had some else broken in
earlier testing.
- In static routes (e.g. default routes) the check-gateway=ping seemed to
be less stable in RouterOS 7 and would drop the route regularly. Needs more investigation.
Conclusions
Based on these experiments, the recommendation is to gradually transition to
RouterOS 7, with the following caveats:
- VPNs on the CCR2004 does not seem to work. This may be a bug in the ARM64
implementation of RouterOS 7. This device requires RouterOS 7 so avoid using
this particular model for now.
- Remote upgrades are always risky and should be avoided. I've been lucky so
far but don't poke the bear. Difficult to access site should probably wait
until next season.
- I have not yet started upgrading radios. Check back here for a
recommendation
- I think RouterOS 7 has definite advantages over RouterOS 6, not even
considering new features such as WireGuard and Zerotier for RouterOS.
However, I still need to convince myself that RouterOS 7 has reached the level
of maturity of RouterOS 6.
So upgrade with care, but remember our motto: D.F.I.U.