Follow

Hi, I'm trying to get a NixOS VM up and running on OpenBSD, but I'm struggling to set up networking.

I'm following the instructions at dataswamp.org/~solene/2021-05-

I followed the steps, but substituted with my IPs.

I used 192.168.0.167/24 in place of 192.168.1.151/24 in the tutorial and for router's IP 192.168.0.1 in place of 192.168.1.254

Unable to access the network. Any pointers will be appreciated.

· · Web · 3 · 2 · 2

Okay, I've found familug.github.io/using-virtua
Will try this in the meantime.

At this point, I would like to get either Alpine Linux or NixOS working.

I want to be able to use /some/ Linux distribution to run certain software that OpenBSD cannot.

@samebchase Here is my full config for a vm running in vmd: gist.github.com/qbit/8322553bc

It builds 2 kernel modules that specifically help with running linux in openbsd's vmm (both modules from dv@).

That doesn't specifically help you with the networking bit, but it will make running the vm less painful later on :D

@samebchase can you ping the OpenBSD host using my setup?

@solene no, but I was able to do the reverse.

For the time being, I tried using an old router to create an ethernet interface by creating a repeater, but this is proving too painful and slow. In the past, I remember that changing the channel caused less interference, but I'm not able to figure out how.

In the FAQ, there are 4 options for networking. openbsd.org/faq/faq16.html#VMM

Assuming, I only have wireless n/w for the time being, shouldn't Option 2 listed there work fine? I'm trying that now.

@solene
gist.github.com/samebchase/992

PF rules are there.

I'm guessing I need to add:

```
match out on egress from 100.64.0.0/10 to any nat-to (egress)
pass in proto { udp tcp } from 100.64.0.0/10 to any port domain \
rdr-to $dns_server port domain
```
to them.

@samebchase if you want to see if pf is making troubles, you can add a pass in quick on tap or disable pf and see if you can ping the host from the VM :)

I don't know your rulesets but you may need to allow some stuff ;)

@solene Okay now I've started with vmctl -L option, and now I am able to ping host from VM, and VM from host using the IP listed for the tap0 interface. 🙏 🤩

Now, I am not able to connect to the internet from inside the VM, so I will add those PF rules given in the FAQ, or just temporarily disable PF altogether, and see if it is working.

@samebchase I'll review my text to see if I made a mistake somewhere

@solene Okay, now I have added the PF rules exactly as given in the FAQ reloaded by pf -f /etc/pf.conf For $dns_server I've given the IP of my router, not sure if that is correct.

enabled IP forwarding by running sysctl net.inet.ip.forwarding=1

Now, I am able to ping the VM from host on 100.64.1.3 and from the VM I am able to ping the tap0 interface on 100.64.1.2 and this is added to /etc/resolv.conf as well.

Current state of files: gist.github.com/samebchase/992

@solene However not able to access the internet. I was briefly able to, but now I am unable to again. :blobpats:

@solene Good news, after doing a pf flush all, I am able to access the internet from inside the VM. Running `nixos-install` now. Hopefully, it's able to fetch all the packages it needs.

@samebchase Nice! What did you do wrong?

Or what was wrong in my guide?

@solene PF flush all did the trick. From what I understand as PF is a stateful Firewall, my hunch is that it got into some intermediate state. (Can this happen? Just a guess from my side...)

I'm using the vanilla configs mentioned by you in your post, and Option 2 mentioned in the networking part of the virtualization FAQ.

My headaches basically were due to not having ethernet, so I can't do the bridge config that your post mentions.

Truly, I am grateful for all the help. 🤩 🙏 2:30 AM now.

@redcepelin @solene
I did `pfctl -F all`.

Is there a better way?

@solene Also, enabling IP forwarding for ipv6 is required. Just simple IP forwarding for ipv4 was not working.

@samebchase @solene -F all removes all rules, reinstall them with pfctl -f /etc/pf.conf afterwards

@samebchase are there any errors you get from the network side? can you post your hostname.bridge0 ? do you have port-forwarding enabled / pf configured? are you able to ping the ip of the host?

@samebchase can you ping the IP of your openbsd machine?

To ping an external-to-the-openbsd machine, you will need to tell openbsd to forward packets:

# sysctl net.inet.ip.forwarding=1

and add a line like below to your pf.conf:

match out on egress inet from !(egress:network) to any nat-to (egress:0)

@qbit @samebchase

"To ping an external-to-the-openbsd machine, you will need to tell openbsd to forward packets:" <= NO, it's not needed, and not necessary!
(and egual for the PF rules)

Host and VM are on a same network!
This case is not a NAT.

@samebchase: check your "networking.nameservers" on your NixOS install! ;)

@hucste @samebchase oh yap, good catch :D

Nameservers wouldn't impact the ability to ping an ip though! :P

@qbit @samebchase

it depends if:
- ping on adress IP: not necessary
- ping on FQDN: it's necessary

@samebchase @qbit

REREAD the official FAQ!!!

- openbsd.org/faq/faq16.html#VMM

and particularly the section "Option 4", where is wroted clarly:

> This option only works for Ethernet-based devices, as the IEEE 802.11 standard prevents wireless interfaces from participating in network bridges.

And segun the showed config, you try to use an WIFI iface! :(

@hucste @samebchase I want to say (maybe incorrectly) that you can use a trunk with wifi+ethernet and it will work

@qbit @samebchase

good idea, but not realistic; Wifi iface can not run into bridge, due to IEEE 802.11 standard.

Into the trunk, if the iface down, the stream network cant be switched to the wifi iface.

@hucste @qbit

Oh wow! Thanks, that's a good catch. :patcat:

Let me try again with a wired interface. Now, I need to find an old router and use it in an extender mode.

Thanks once again for all the pointers. Will try with a wired interface shortly.

@hucste @qbit

I'm using a wired interface now.

I've reached a point where I can ping and ssh into the VM from the host, but I cannot ping the host from the VM, nor can I access the internet.

@hucste @qbit I'm using gist.github.com/samebchase/992 as my latest config.

I'm able to access the internet from within the VM now. Thanks for all the help yesterday! Otherwise I would have never known that WiFi is a no-go as a bridge. 🙏

@samebchase @qbit

With this config, you access to the Net? amazing.

OK, you translate (NAT); sincerly beurk!

the finals PF rules "('match out …' and 'pass in proto { tcp udp } from 100.64.0.0/10 …)" are not really usefull.

You are not on Bridge NAT!

---

Normally, you resolv.conf on your Nixos machine contain
only the adress ip of DNS name server!
for instance: 8.8.8.8
(or another…)

And your gateway on Nixos machine must be your gateway on your segment network 192.168.0.0!

@hucste @qbit These settings are directly from openbsd.org/faq/faq16.html#VMM

Over there I can see 4 options to get networking enabled in the VM.

Out of that, I cannot use option 4 because I don't have a wired ethernet connection at home, as my wireless router is too far away.

I took the config that they have given for Option 2, and it seems to be working now.

I'll take a look at your article and read further. I'm sure this setup could be cleaned up, as I learn more about the various components.

@samebchase @qbit

OK, I understand your problem.

"Bad situation" results bad config network!

Have a good day! :D

@samebchase @qbit

OK. Instead of using "Option 4", in your case, config your VM segun "Option 3"
This use too bridge, with vether iface, (and not directly your iface ethernet).

In the exemple, on the VM FAQ, it use 10.0.0.0/8 subnet, but for your needed (@home), you can config 192.168.1.0/24 network;
- set 192.168.1.1 as addr ip on the vether iface
- and on resolv.conf nixos machine, set your vm as 192.168.1.1

And for the PF rules, just write segun VM FAQ Option 3, and not Option 2!

@samebchase @qbit

fixe:
- on resolv.conf nixos machine, set your gw (gateway; not vm): 192.168.1.1

@samebchase @qbit

So you can have your VMs working.

Your Wifi or ethernet iface will not be directly managed by the bridge.
the network flow arriving on the wifi interface will be able to switch to the bridge without further modification.

Excuse. I'm tired:
- fixe on the VM (Nixos):
on the resolv.conf: set an dns ip address, like 9.9.9.9
and the gw: the adress ip of your vether iface (as, 192.168.1.1)

understands the principle and adjusts it to the configuration

:D

@samebchase @qbit

Why choosing private network Class C, more than Class A?
Because, on your PC, @home,
you will not have more than 254 virtual machines.
Configure with private Class C is enough! :p

@hucste @qbit

Hey, thanks so much for the help. I will spend some time learning all these things and optimize my config in the manner you suggested.

You're right, there's no way I'll need more than 254. Class C should be enough for me.

@hucste @qbit I have changed the config to the one recommended by you. I'm on a Class C network now, and I've changed the PF configuration to what is described in Option 3, in the Virtualization network FAQ.

It's working well now. Thanks once again, I learnt a lot of new stuff trying to set up all this.

Show newer

@hucste @samebchase Sure, but in the gist samebchase is pinging an ip :P

@qbit @samebchase

samel, for ping, prefers to use the option -c ;-)

ping -c3 address_ip

one ping is not enough to ensure proprer operation. :p

Sign in to participate in the conversation
Mastodon

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!