IOT Monitoring / IDS Home Server

Visit Repo

This project is a simple home server I had made to monitor IOT devices in my home network for their downtime, as well as catching any possible intrusions into the network. Primarily being a proof of concept, there are some limitations to the product, which I will share about as well. This project is especially helpful in a scenario where there is a lack of admin controls to the router on the network.

Breakdown¶

Here is the architecture of the project

The entire process is hosted on a Raspberry Pi 4 Model B with 8GB of ram. Grafana is an open source analytics and interactive visualization web application, while Loki is a log aggregation system to store and query logs from applications. This repo assumes that Nmap, Grafana and Loki clients have been setup on the machine.

A python script first executes an Nmap command in a 5 mins interval
- A Samba share is configured to allow me to modify the scripts through my home computer
- The following Nmap command is used, the parameter -sn disbales port scan as we only want to know the existence of the devices of the network, while running the command in sudo allows for us to run in privilege mode, getting more info such as the MAC address of the devices:
```
sudo nmap -sn 192.168.0.0/24
```

The script retrieves the list of devices, every device being in the following format:

Nmap scan report for L920.Midkemia (192.168.0.70)
Host is up (0.030s latency).
MAC Address: 9C:53:22:3E:E9:44 (Unknown)

While there is an existing solution to use a Promtail client to push logs generated by my python script to Loki, I decided to go with python-logging-loki, which wraps around the exisitng logging library in python and sends logs straight to Loki with the Loki HTTP API
Grafana then pulls the logs stored in Loki to be visualised in dashboards and create alerts.

Dashboard Preview¶

1. Top half - Bedroom Lights¶

From the list of devices retrieved in the automated Nmap scan, Grafana identifies if a list of MAC addresses are present in the scan. These MAC addresses are that of my Wifi-controlled bedroom lights (6 of them). It's absence indicates that the device is experienced a timeout and may not be responsive.

In my case, this section is useful as I occasionally experience my lights being unresponsive to commands such as colour change or switching on/off.

2. Bottom half - Nmap Sweeper¶

More metrics is displayed in this section, showing a trend of the number of devices detected in the network over time, as well the number of outliers.

In this case, an outlier refers to a new device detected in the scan that was not present in the previous scan, it can be an indication to new joiners in the network. However, due to the inconsistency in the scan (timeout when trying to reach existing devices etc), better methods should be used to detect outliers to avoid false positives.

Future Improvements¶

1. Outlier detection¶

Instead of detecting outliers by finding devices that were absent from the previous scan interval, more robust solutions should be used to detect the outlier based off a trend of devices staying on the network in the past few scan intervals.

Retaining a list of trusted devices will definitely be helpful in the detection as well.

2. Alerts¶

When looking through high volumes of log data, we can overlook abnormal activities such as an untrusted device in the network. In that case, we can create alert rules with Grafana where our admin can be informed of new activities in the network via email or other available contact points.

Other Useful Resources¶

Creating Daemon Processes (Automation)¶

This Medium article provides a tutorial on setting up a python script as a service through systemctl/systemd

Loki "Too Many Outstanding Requests"¶

This is a common error when trying to query Loki for logs data. A fix can be seen in Stack Overflow.

Samba Permission Denied¶

There was an issue encountered where the Linux permissions conflicted with the Samba user access. This can be fixed through the steps in a forum here.

Entering Virtualenv via Bash script¶

In the start.sh script, before the python script is executed, the script first enters the python virtual environment to access the required modules in the python script.

Entering venv in CLI

However, when doing it in a Bash script, specify #!/bin/bash in the first line. The final script should look like this:

#!/bin/bash

python3 -m venv /path/to/ids/.venv
source /path/to/ids/.venv/bin/activate
python3 /path/to/ids/nmap.py