Oof! A project I've been working on for about a year at $DAYJOB got finally released as FLOSS:
The basic idea is:
1. get your DNS zone file
2. set up some rules for each host
3. deploy this to a small VM somewhere
4. check the generated test results every day to see if there are any surprisingly open ports, hosts that should not be up/pingable, or #TLS config that is less than great on some endpoints
I need to document it a bit better, yes.
@rysiek Wow interesting. I'm trying to wrap my head around it. It's like a tripwire setup for external monitoring of networks and domains?
I love that you did it yourself but just to understand, this could be done with Shodan right? But it would most likely require a subscription. But this all seems like something Shodan already scans for. Open ports, services, names, TLS issuers.
@stemid yes, Shodan, BitSight, and other similar services scan for that. BitSight doesn't tell you *when* it scans and doesn't give you the raw scan results. Shodan requires subscription.
This is not as advanced as Shodan, but it gets the job done for us. It performs scans on a regular schedule, and we get to have full raw scan data, plus know exactly when a thing got scanned. It's also easily integrated with Zabbix, and Gitlab CI/CD.
There's plenty to improve, of course.
@stemid to be fair, I'd need to check Shodan out, haven't played with it for ages. Thanks for reminding me of it!
Prober lets you configure what you test for pretty exactly (and it will become even more exact). Plus, you get to configure it using YAML, which you can (and we do) keep it in git for easier management.
Finally, if you're running it yourself, you're the only one who has access to the zone configs and test results. That might or might not be something you care about.
@stemid oh, and another thing - not sure Shodan lets you deploy in specific places or parts of your infra.
Prober can be deployed somewhere outside of the infrastructure that is being checked, or anywhere inside of it, so that you can check different assumptions from different vantage points. For example, "jenkins.example.com should have ports 22/tcp and 443/tcp open from internal infra, only 443/tcp open from the DMZ, and all filtered from the external network".
@stemid thanks for asking, by the way, it got me to think about it and write down the actual differences.
@rysiek Well I think the main difference is that you're doing it yourself with the freedom to host anywhere while Shodan most likely use their own infrastructure. Perhaps they offer some probe software you can run locally, I have not heard of anything like that. When it comes to security hosting it yourself is often important to people.
To be quite honest, I am oscillating between "hey, I think it might be a useful thing" and "it's basically poor man's Shodan, and I should be ashamed of myself for wasting people's time", but leaning towards the former currently.
@rysiek SHODAN doesn't often scan inside networks. And there are times where SHODAN is outside of the toolset one can use.
I say, rock on.
Server run by the main developers of the project It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!