So you're out looking for a VPN for all your networking needs and somehow stumbled across this page. Well let me present Nebula to you!
Nebula is an overlay network which allows you to join machines into a single virtual network. It is also incredibly simple to set up and understand. The underlying cryptographical protocol is the same as WireGuard (Noise Protocol Framework) with the advantage of being easier to manage.
This article will explain the core concepts behind Nebulas architecture and get you started in setting up your own network. I will also compare it to WireGuard in some places, and elaborate why you would want to use Nebula in its place.
If you already know what a VPN is (not the YouTube sponsor kind), feel free to skip this section
A VPN (Virtual Private Network) is a rather loosely defined term in the space of network engineering. On the most basic level, it provides a way to interconnect multiple machines without having to physically wire them together.
Even from this basic example, multiple use cases arise:
- Connecting geographically distributed datacenters
- Connecting clients to a protected network
- Tunneling internet traffic through a designated gateway or firewall
This last use case has become the most widely spread meaning for VPN. It is also possible to build something like this using Nebula, however the focus of this article lies in the use case of connecting multiple datacenters or clouds.
From a technical perspective, VPNs work by encapsulating network packets on a higher level. To facilitate this, a network device is created and configured using routing rules. Whenever the interface receives a packet from the host, it encapsulates it into its own application level data packet and sends it of to its destination. The destination has to run the VPN software as well. On the receiving side, the packet is unpacked and sent from the virtual interface to the operating system.
This of course results in an overhead on top of regular IP communication but in most cases, this performance penalty is negligible.
How Nebula works
Nebula is a so called Mesh Network in which nodes talk directly to each other without passing a central gateway. This works by clever exploitation of networking properties.
A regular networking flow, which happens thousands of times per day looks (absurdly simplified) like this:
When establishing a connection, the client opens a port for responses from the server. Firewalls and routers between the client and server know about this process and ensure that traffic back to the client is permitted.
Nebula uses this open port to cleverly traverse firewalls and connect clients directly to each other. A special node, called Lighthouse in nebula terminology, is used to have clients discover each other. This special node is the only network member which has to have a publicly reachable IP address. The client first connects to the lighthouse and opens a local port. The lighthouse however, does not directly respond but rather instructs the target client to respond on the newly opened port. This process is repeated for the other side as well.
Please note that this is a very simplified diagram. In reality, a lot more spoofing and trickery is going on to enable seamless NAT traversal.
One big advantage over WireGuard, is the way Nebula handles authentication. WireGuard expects the administrator to integrate the public key of new clients into the configuration of each existing node. While being simple in theory, this approach does not scale very well and requires additional tooling to automatically roll out new nodes. Tailscale and consorts are good ways to manage this complexity but come at the price of introducing additional services and abstractions.
Nebula utilizes Public Key Infrastructure (PKI) concepts to address this issue. A central certificate authority (CA) signs certificates of clients. These certificates are used for cryptographic purposes as well as administrative ones. The signature contains the IP address of the node, as well as other metadata like allowed subnets and groups.
Whenever a new client registers with the lighthouse, it verifies the certificate with the certificate authority and remembers the IP address stored in the certificate. Since the CA itself does not change, the lighthouse can verify clients which have been created after it has been configured, and it does not need to be adapted to these changes.
Our example setup
The setup we're discussing in this article is as follows:
earth: On-premise server, located at my house, connected to the internet via a consumer ISP
moon: VPS located in a nearby datacenter. Has its own public IP and will be our lighthouse
jupiter[01:04]: VPS running on a cloud provider. They do not have permanent public IP addresses
If you want to follow along, you can use any machines you have access to. Google Cloud, Azure, AWS and Oracle Cloud all have free tiers for you to play around with.
Our goal is to have each machine on the same network, for reasons coming up in a future blog post.
Setting up Nebula
Nebula is provided as a statically compiled binary and can be downloaded on the
official GitHub releases page.
The release contains two important files:
nebula-cert. To install
these, please follow the usual procedure for your OS. Most likely you will want
to these two binaries to a directory contained in your
Our first step is to set up the certificate authority:
nebula-cert ca -name univer.ze
This is the minimal command creates a new certificate authority with the name
univer.ze. This CA on its own does not do very much. Make sure to keep the
ca.crt in a safe place as these files will allow you to onboard new
clients onto your network. At the creation stage, it is also possible to
restrict a few variables like IP subnets or groups. For this example, a minimal
CA allows for the most flexibility.
Now onto registering the clients. Let's start with the lighthouse aka
nebula-cert sign \ -name "moon.univer.ze" \ -ip "172.30.42.1/24" \ -out-crt "moon.crt" \ -out-key "moon.key" \ -groups "trusted"
This command contains a lot of information. Here's a breakdown:
|name||Friendly name of the client. Does not have to be the actual hostname of the target machine|
|ip||Designated IP address inside the VPN. You can freely choose any address here, but you have to keep track of the addresses yourself|
|out-crt||Output location for the certificate|
|out-key||Output location for the key|
|groups||Client groups. This is used for advanced firewall rules|
Make sure you execute the command in the same directory as the
files or specify these files using the
If you provide services to external users and want to introduce an additional
layer of security, you can have them send you a public certificate generated by
nebula-cert keygen and sign it on your machine. This way, you never see the
After setting up the certificates for the lighthouse, we have to perform the same procedure for the clients.
Now that our certificates are ready, we can begin configuring the machines. The
first step is to download and install the
nebula binary on the system. The
second component is the configuration file.
Nebula is configured using a
yaml file and an example can be found
repository. This example is very well documented and describes specific
use cases for most options. Below is minimum configuration for our setup, but I
urge you to read the configuration file yourself.
# Configuration file for the lighthouse moon.univer.ze --- pki: ca: /etc/nebula/ca.crt cert: /etc/nebula/moon.crt key: /etc/nebula/moon.key lighthouse: am_lighthouse: true listen: host: "[::]" port: 4242 tun: dev: nebula1 firewall: outbound: - port: any proto: any host: any inbound: - port: any proto: any groups: - trusted
If our lighthouse should not be a member itself, the
tun interface can also be
disabled. In this mode, the lighthouse is only used for discovery.
The firewall section can be used to restrict incoming and outgoing traffic. This
is also the reason one can assign groups to hosts during the signing process.
For this example, we allow all outbound traffic and only allow inbound traffic
To run the lighthouse, execute
nebula -config config.yaml as a user with enough
access rights. Templates for service managers can be found here.
Pro Tip: If you add the
cap_net_adminto the nebula executable, Nebula can be run as any user. The command to add this capability is:
setcap cap_net_admin+ep nebula
Now that the lighthouse is up and running, let's take a look at the client configuration file:
# Configuration file for the client earth.univer.ze --- pki: ca: /etc/nebula/ca.crt cert: /etc/nebula/host.crt key: /etc/nebula/host.key static_host_map: "172.30.42.1": ["203.0.113.42:4242"] lighthouse: am_lighthouse: false hosts: - '172.30.42.1' listen: host: '0.0.0.0' port: 0 punchy: punch: true tun: dev: nebula1 firewall: outbound: - port: any proto: any host: any inbound: - port: any proto: any groups: - trusted
While most sections are pretty straight forward, the
some further explanation.
203.0.113.42 is the publicly routable IP address of
172.30.42.1 its internal one. The static host map is used to create a
mapping between internal and external addresses. In our case, only the
lighthouse IP has to be entered here. If multiple machines are reachable from
the public internet, we could enter them here to gain a small performance boost
as the lighthouse is not needed for the link setup.
In the lighthouse section, we can specify any number of lighthouse addresses. It
is important to enter the internal address here rather than the external one.
The mapping between internal and external addresses happens in the
Two other interesting things happen in this config. The special port value
tells Nebula that this is a roaming node and dynamic port assignments shall be
used. Another related setting is
punchy. Behind this innocent name lies a
subsystem designed to keep firewall NAT mappings alive by periodically "punching"
new connections to the firewall. Further tuning options are available for more
complex NAT setups.
Running the client works the same way as starting the lighthouse server:
nebula -config config.yaml
And just like that, the two hosts are able to reach each other!
[[email protected] ~]$ ping 172.30.42.1 PING 172.30.42.1 (172.30.42.1) 56(84) bytes of data. 64 bytes from 172.30.42.1: icmp_seq=1 ttl=64 time=0.449 ms [[email protected] ~]$ ping 172.30.42.2 PING 172.30.42.2 (172.30.42.2) 56(84) bytes of data. 64 bytes from 172.30.42.2: icmp_seq=1 ttl=64 time=0.683 ms
The configuration for the
jupiter nodes is exactly the same, and left out in
this article for brevity.
Now that our network is up and running, we can see the mesh in action. Communication between the jupiter nodes is much faster and has lower latency while communication across the internet takes a bit more time. In a classical VPN setup, all traffic, regardless of hierarchy, would have gone through the lighthouse which would further slow down communications.
Of course there are a lot of more things you can do with Nebula (for example adding non-Nebula hosts) but this post should be enough to get you started and figure out the rest for yourself. Hopefully I convinced you to try out Nebula for yourself and simplify your network.