My Blog

Made lots of minor changes to the ‘box’ recently while recovering from hip replacement.

Zigbee2mqtt

• Updated to version 2.0.0 which needed a fix to the configuration.yaml
• Still using usbip to contact the zigbee controller on the Pi running on 192.xxx.xxx.220
• can use this
  ls -l /dev/serial/by-id
  to discover the location of the controller on the OMV server pointing to the Pi.  Pi uses usbipd service and using
  lsusb
  useful in location of the zigbee controller.

Find files of certain size

find .m4v -size +200M

find files by name and move them

sudo find -name *.m4v -exec mv “{}” /media/6f326f9f-b1a4-484c-b6d6-e8768b9eee3e \;

size can be added too…..

sudo find -name *.m4v -size +200M -exec mv “{}” /media/6f326f9f-b1a4-484c-b6d6-e8768b9eee3e \;

Ports in use

This command useful
sudo lsof -i:ppp
where ppp is the port e.g. sudo lsof -i:443

Stop mails in crontab

put MAILTO=”” infront to the the cron

Stop things writing to the syslog

1.  owlmqtt service – commented out the echo
2. crontab as above of the Cron running every 5 mins from NextCloud
3. removed checking of ssh from monit as this was causing an entry from 127.0.0.1 in the syslog every time it was checked.

run command inside docker container

sudo docker exec -u root -it dashy /bin/bash -c ‘yarn validate-config’

/bin/bash might be /bin/sh depending on the container

look for string in folder and lower files

grep -r . -e “string”

Added to Docker-Compose

1. New version of bitwarden
2. portainer
3. watchtower
4. vaultwarden – to access old bitwarden database
5. plex

Reinstalled

1. beets

OMV

Salt

needed to move some salt python modules on the installed pyenv to the local python3 libraries used from sudo.  Tried to do this properly and gave up after 2 days  Needed to be able to run omv-salt command

removed k3s service – Kubernet was a plug in installed and removed but left the service behind

remove podman service – Podman was a plug in installed and removed but left the service behind

omv-extras

sudo -u root wget -O – https://github.com/OpenMediaVault-Plugin-Developers/packages/raw/master/install | bash

OMV notifications

used one of the email addresses for zen after trying to use SMTP for both google mail and outlook and finding they no longer support simple SMTP authentication!!

OMV – run the update which is done after a change in the command line

useful when it fails and you want to ‘watch’ the fail.

sudo omv-rpc -u admin “Config” “applyChanges” “{\”modules\”: $(cat /var/lib/openmediavault/dirtymodules.json), \”force\”: true}”

php

Tidied up the various php services running.  Stupidly remove php-pam which mucked up the login for openmediavault and required an omv remove and install.  reinstall did not fix.

nginx and certbot

Updating certificates

use command below (-v is verbose and useful)

sudo certbot –nginx -v

This exists as a paid service but wanted to see whether it would work for any of the following

Mega
Mediafire
pCloud
Dropbox
Evernote

It turns out that Mega is by far the best but given the age of my Synology box there is no package solution but there is a scripting language.

In the case of the others e.g. pCloud and Mediafire there is nothing but drag and drop.

I kept on finding there were loads of refresh tokens when logging into HA. So I googled why….. I am not logging out! No surprise there but they are a pain to delete individually and so I found this page.

Delete old/abandoned refresh tokens from user profiles

The tokens are in the .homeassistant/.storage/auth file and held as JSON.

The link explains that something call ‘jq’ can edit JSON from the command line… neat! However I had trouble getting the script in the link to work and so created a new shell script like this….

authfile=/home/homeassistant/.homeassistant/.storage/auth
tmp=”$(tempfile)”
echo “Copying from ” $authfile ” to ” $tmp
jq –arg s “$(date -d “” +”%Y-%m-%dT%H:%M”)” ‘del( .data.refresh_tokens[] | select(.last_used_at < $s) )’ $authfile >$tmp
cp $tmp $authfile
rm $tmp

This needs to be run as the homeassistant user to work and homeassistant needs to be stopped as the auth file seems to be loaded into memory when ha is running.

What it does is call jq after setting up the auth file location and a system temp file using –args. –args passes the ‘s’ as a string of value “$(date -d “”+…) into the command ‘del(.data….) using the $authfile as input. The command deletes the part of the JSON file with the refresh token where the last used date is less than today and copies it to the temp file.

When this is complete the temp file is copied over the authfile and then the temp is deleted.

Works well. Just need to remember to restart home assistant!

Getting Graphite Running

After having installed Icinga2, IcingaWeb and IcingaDirector I decided I wanted some better pretty pictures and graphs.

I have Grafana installed via HomeAssistant and so decided the next point was to install Graphite and get some data into it and then to Grafana for display.

It was not as easy as expected.

First port of call to see how to do this was https://graphite.readthedocs.io/en/latest/index.html which is IMHO not that easy to follow – it is all there but the actual steps are not that clear and initially I got stuck at Initial Configuration and the Nginx + uWSGI step and gave up and deleted everything including the virtual environment I had created.

I then decided to start again and read the dependencies a bit more carefully and also looked here https://gist.github.com/tristanbes/4046457 and here https://arpitbhayani.me/blogs/setting-up-graphite-using-nginx-on-ubuntu.

So I started again….

• No virtual environment
• Python3
• Now the dependencies ….
  • Django (I had heard of this but did not know what it was). The tutorial here is great …. https://uwsgi-docs.readthedocs.io/en/latest/tutorials/Django_and_nginx.html and I followed this and got the sample app running. Main problems were getting the user and group www-data:www-data the right permissions.
  • I also tried to install django-tagging using pip3 and found it was already there as were these dependencies – putz and scandir
  • I tried to install fontconfig using pip3 and it failed and with a bit of googling found out this was a apt install apt-get install -y fontconfig. This was also up to date.
• Then back to the instructions here https://graphite.readthedocs.io/en/latest/index.html and decided on default layout without a virtual environment.
• Checked with development headers – all OK now. apt-get install python-dev libcairo2-dev libffi-dev build-essential
• Installed packages
  • export PYTHONPATH=”/opt/graphite/lib/:/opt/graphite/webapp/”
  • pip install –no-binary=:all: https://github.com/graphite-project/whisper/tarball/master
  • pip install –no-binary=:all: https://github.com/graphite-project/carbon/tarball/master
  • pip install –no-binary=:all: https://github.com/graphite-project/graphite-web/tarball/master
• From the link a arpitbhayani.me I decided to use Postgres as the database and followed his config instructions and so needed to sudo pip3 install psycopg2 as psycopg2 is a dependency for using Postgres.
• Then did the Webapp Database setup – this is why Django was needed to be installed BEFORE this step – i.e. dependency
• Then the steps in Nginx + uWSGI were followed… with a few changes
  • I decided to use a socket for NGINX and so the graphite.ini was different (socket = /opt/graphite/webapp/graphite/graphite.sock) and as I have no virtual environment the virtualenv setting not needed.
• The Nginx conf file is as follows….

# /etc/nginx/sites-available/graphite.conf
upstream graphite {
    #server unix:///path/to/your/mysite/mysite.sock; # for a file socket
    server unix:///opt/graphite/webapp/graphite/graphite.sock; # for a file socket

}

server {
    listen 8080;
   # the domain name it will serve for
    server_name <ip-address>; # substitute your machine's IP address or FQDN
    charset     utf-8;

    location /static/ {
        alias /opt/graphite/webapp/content/;
    }

    location / {
        include /opt/graphite/conf/uwsgi_params;
        uwsgi_pass graphite;
    }
}

• uWSGI Emperor was used to get the link between uWSGI and NGINX (see here https://uwsgi-docs.readthedocs.io/en/latest/tutorials/Django_and_nginx.html for the setup)
• Continue the config with uncommenting …. from graphite.app_settings import * at the bottom of the local_settings.py file.

Configuring Carbon

• Firstly copy some files from /opt/graphite/conf for editing (now only at line 18 and 19 in this link https://gist.github.com/tristanbes/4046457)
  • sudo cp carbon.conf.example carbon.conf
  • sudo cp storage-schemas.conf.example storage-schemas.conf
• and now do lines 21-56 in the link from tristanbes to set up storage-schemas.conf and storage-aggregation.conf
• Next following the instructions on editing the carbon.conf in the main graphite documentation including changing the cache ports 2003 to 2013 and 2003 to 2014 in cache section and the Destination setting in relay section
• Now set up a system service for carbon-cache

[Unit]
Description=Graphite Carbon Cache
After=network.target

[Service]
Type=forking
StandardOutput=syslog
StandardError=syslog
ExecStart=/opt/graphite/bin/carbon-cache.py –config=/opt/graphite/conf/carbon.conf –pidfile=/var/run/carbon-cache.pid start
ExecReload=/bin/kill -USR1 $MAINPID
PIDFile=/var/run/carbon-cache.pid

[Install]
WantedBy=multi-user.target

• enable the service (sudo systemctl enable carbon-cache.service)
• and start it.
• Try sending some into port 2013.
• e.g.

echo "test.count 9 `date +%s`" | nc -q0 127.0.0.1 2003;

This will add one data metric of value 9 in system. Lets add some more data; this time wee loop through values

for i in 4 6 8 16 2; do echo "test.count $i `date +%s`" | nc -q0 127.0.0.1 2003; sleep 6; done

• and you should be able to see data in Graphite like this

I struggled with the LetsEncrypt certificate renewal using the plugin in OMV as I have a separate sites.enabled running in parallel.

To do this in the future I need to login into webmin and disable www.thenaylors.co.uk

then go into OMV and stop SSL and run only on port 80.

Then do the update of the certificates and then

rerun SSL on OMV and then re-enable www.thenaylors.co.uk on webmin.