Updating NextCloud with the Open Updater button does not work

Updated the items below as not working. Much easier to navigate to the nextcloud folder and execute

sudo -u www-data php updater/updater.phar

————-

If the “Open Updater” button does not work then try this.

Go to /updater/login.php

Open terminal window and execute php -r ‘$password = trim(shell_exec(“openssl rand -base64 48”));if(strlen($password) === 64) {$hash = password_hash($password, PASSWORD_DEFAULT) . “\n”; echo “Insert as \”updater.secret\”: “.$hash; echo “The plaintext value is: “.$password.”\n”;}else{echo “Could not execute OpenSSL.\n”;};’

Check that the updater.secret is as the hashed version in the config.php file for next cloud.

Enter the non hashed version as the password if requested on the login.php page.

It should then work.

It is also possible to run this manually from the next cloud home directory like this …..

sudo -u www-data php /media/www/nextcloud/updater/updater.phar
sudo -u www-data php occ upgrade
sudo -u www-data php occ maintenance:mode –off

Upgrading Home Assistant

  1. Stop Home Assistant (sudo systemctl stop homeassistant.service)
  2. Open the directory where the virtual environment is located: (cd /srv/homeassistant/)
  3. login as homeassistant…. sudo -u homeassistant -H -s
  4. Activate the virtual environment: source bin/activate
  5. Upgrade Home Assistant: ‘python3 -m pip install –upgrade homeassistant’
  6. Start Home Assistant (sudo systemctl start homeassistant.service)
  7. Check status (sudo systemctl status homeassistant.service)
  8. You can now reach the web interface on http://ipaddress:8123/ – the first start may take up to 20 minutes before the web interface is available

Update

  • If python needs updating….
    • As pyenv is installed a download of python and
      • a ./configure – -enable-optimisations followed by a make -j2 does not work as a module called encodings is not found.
      • there is another way ……
    • Create a new venv using the pyenv version of python
      1. get the latest version of python from pyenv needed.  
        • pyenv list
        • pyenv install <version>
        • pyenv global <version>
      2. stop homeassistant. sudo systemctl stop homeassistant
      3. login as homeassistant
      4. Go to /srv …. cd /srv
      5. sudo mv homeassistant homeassistant_old_<yymmdd>
      6. sudo mkdir homeassistant
      7. change ownership. sudo crown homeassistant:users homeassistant
      8. now install virtual environment.
        • cd /homeassistant
        • python13 -m venv .
        • source bin/activate
      9. Install homeassistant.  python3 -m pip install homeassistant
      10. system runs mysql database and this needs to be installed as well python3 -m pip install mysql
      11. check has installed.  file ‘hass’ should be in /srv/homeassistant/bin
      12. restart.  sudo systemctl restart homeassistant

Installing the HADashboard on Home Assistant

After having used Home Assistant  for a few weeks I wondered whether these was a “nicer” front end or dashboard.  A bit of Googling and I found this…. https://www.home-assistant.io/docs/ecosystem/hadashboard/.

This takes you to the install for Appdaemon here https://appdaemon.readthedocs.io/en/stable/DASHBOARD_INSTALL.html.  This then directs you to the install site here https://www.home-assistant.io/docs/configuration/packages/.

Installing Appdaemon

The instructions are pretty clear to start with.

1.  Install appdaemon using pip3 ($ sudo pip3 install appdaemon)

but then it gets a little confusing (well to me) as it starts discussing development builds etc and then goes down to running the app and indicates a link to /home/homeassistant/conf.
Well on checking I do not have that directory.

So you then need to go back to the 2nd link I posted above. https://appdaemon.readthedocs.io/en/stable/DASHBOARD_INSTALL.html and look firther down the are some instructions in a different section called configuration.  https://appdaemon.readthedocs.io/en/stable/CONFIGURE.html follow the instructions there.

  1.  Set up a new directory /home/homeassistant/conf
  2. Create appdaemon.yaml and populate this
  3. Go to Home Assistant and create a Long Lived Access Token.  The instructions on how to do this are on the same page but at the bottom
  4. Update the appdaemon.yaml with this token
  5. Skips over the section on Filters (I guess in the future I may come back to this)
  6. MQTT is added
  7. Test App added

Back to the installation instructions and changes made to get the service starting at boot time.

  • $ sudo nano /etc/systemd/system/appdaemon@appdaemon.service
  • Create the file replacing the user bit %I and the <full path to config directory>
[Unit]
Description=AppDaemon
After=home-assistant@homeassistant.service
[Service]
Type=simple
User=%I
ExecStart=/usr/local/bin/appdaemon -c <full path to config directory>
[Install]
WantedBy=multi-user.target
  • $ sudo systemctl daemon-reload
  • $ sudo systemctl enable appdaemon@appdaemon.service –now
  • $ sudosystemctl start appdaemon@appdaemon.service

Installing the Dashboard

Now back here https://appdaemon.readthedocs.io/en/stable/DASHBOARD_INSTALL.html.

The appdaemon.yaml file need updating again.  This time with

hadashboard:  
dash_url: http://192.168.1.20:5050
dash_ssl_certificate: /etc/letsencrypt/live/somehost/fullchain.pem
dash_ssl_key: /etc/letsencrypt/live/somehost/privkey.pem
dash_password: !secret dash_password
dashboard_dir: /home/homeassistant/conf/dashboards

Add a new directory /home/homeassistant/conf/dashboards and create a new file hello.dash and add this to it:

 

##
## Main arguments, all optional
##
title: Hello Panel
widget_dimensions: [120, 120]
widget_margins: [5, 5]
columns: 8

label:
    widget_type: label
    text: Hello World

layout:
    - label(2x2)

You can now navigate to the url e.g. http://192.168.1.20:5050.

Owl Intuition, logging to Mosquitto and Home Assistant

So thanks to Google again and the Home Assistant wiki I now have the Owl Intuition Solar readings being sent to Mosquitto and used as a series of sensors in Home Assistant.

Starting point was this wiki entry and the entry by jchasey. https://community.home-assistant.io/t/owl-intuition-pv-home-assistant/18157/3.

The python script was amended to take solar reading and not hot_water and heating.

import socket
import struct
import json
from xml.etree import ElementTree as ET

OWL_PORT = 22600
OWL_GROUP = "224.192.32.19"

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((OWL_GROUP, OWL_PORT))
mreq = struct.pack("=4sl", socket.inet_aton(OWL_GROUP), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)

while True:
    # Collect the XML multicast message
    xml, addr = sock.recvfrom(1024)
    # Parse the XML string
    root = ET.fromstring(xml)

    # print (root)

    # Only process those messages we are interested in

    if root.tag == 'electricity':
      timestamp_value = 0
      timestamp = root.find('timestamp')
      if timestamp is not None:
           timestamp_value = int(timestamp.text)
           #print ("time", timestamp_value)
        
      signal_rssi_value = 0
      signal_lqi_value = 0
      signal = root.find('signal')
      if signal is not None:
        signal_rssi_value = int(signal.attrib["rssi"])
        signal_lqi_value = int(signal.attrib["lqi"])
        #print ("rssi", signal_rssi_value)
        #print ("lqi", signal_lqi_value)

      battery_value = 0.0
      battery = root.find('battery')
      if battery is not None:
        battery_value = float(battery.attrib["level"].rstrip("%"))
        #print ("battery value", battery_value)

      for chan in root.iter('chan'):
        #print (chan.attrib)
        
        chanid = chan.get('id')
        #print ("chandid", chanid)
        if chanid == "0":
          current_now = 0.0
          current = chan.find('curr')
          if current is not None:
              current_now = float(current.text)
              #print ("current now", current_now)
          current_day = 0.0
          current = chan.find('day')
          if current is not None:
              current_day = float(current.text)
              #print ("current day", current_day)

        if chanid == "1":
          solar_now = 0.0
          solar = chan.find('curr')
          if solar is not None:
              solar_now = float(solar.text)
              #print ("solar now", solar_now)
          solar_day = 0.0
          solar = chan.find('day')
          if solar is not None:
              solar_day = float(solar.text)
              #print ("solar day", solar_day)

    if root.tag == 'solar':
        solar_exported = 0.0
        solar = root.find('day/exported')
        if solar is not None:
            solar_exported = float(solar.text)
            #print ("solar export", solar_exported)
            print (json.dumps({'type': root.tag, \
                       'timestamp': timestamp_value, \
                       'battery_level': battery_value, \
                       'current_now': current_now, \
                       'current_day': current_day,\
                       'solar_now': solar_now,\
                       'solar_day': solar_day,\
                       'solar_exported': solar_exported}))
            

I then added a shell script to run the python.  This was placed into /usr/local/bin with permission at 0400.

#!/bin/sh
#
# owl_mcast2mqtt.sh
#
# Publish json output of the owl_mcast2json script to MQTT 
#
export LANG=C
PATH="/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin"

MQTT_HOST="ip.address.0.0"
MQTT_PORT="8883"
MQTT_USER="xxx"
MQTT_PASS="xxx"

echo "OWL Network Multicast 2 MQTT"

python3 -u /usr/owlintuition/owl.py | while read line
do
	MQTT_TOPIC="tele/home/mcast/$(echo $line | jq --raw-output '.type')"
	# Publish the json to the appropriate topic
	echo $line | mosquitto_pub -h $MQTT_HOST -p $MQTT_PORT -u $MQTT_USER -P $MQTT_PASS -i OWL_MCast -r -l -t $MQTT_TOPIC
done

A service was then created and enabled.

[Unit]
Description=OWL MQTT store
After=suspend.target

[Service]
User=root
Type=oneshot
ExecStart=/usr/local/bin/owl_mcast2mqtt.sh
TimeoutSec=0
StandardOutput=syslog

[Install]
WantedBy=suspend.target

Strangely after a power cut all the above “disappeared”.

Mosquito needed reinstalling – link here https://www.switchdoc.com/2018/02/tutorial-installing-and-testing-mosquitto-mqtt-on-raspberry-pi/

jq needed installing – link here https://stedolan.github.io/jq/download/

This is a useful reminder for getting scripts working – https://www.raspberrypi.org/forums/viewtopic.php?t=197513

Update to NGINX config for OpenMediaVault stops sites working … the fix.

Tried to access the sites I had set up via https://myurl/xxxx and they were not working today.

Checking the /etc/nginx/sites-available I could see that an update had been done on the 23 Oct and further checking shows this seemed to be to the openmediavault-webgui.

Checking the backup made a few weeks ago showed that the
listen [::]:80 ipv6only=off;
line had been changed to
listen 80;
and
listen [::]:443 ipv6only=off;
changed to
listen 443;
changing back seems to fix everything!
Not sure why right now.
The net result of this was that the 2nd server block I had for myurl was being ignored by NGINX.
Looks like I might need to fix this regularly from now on which is a bit of a pain!

Getting a Raspberry Pi to operate an old Bye Bye Standby (BBSB) RF 433 switch

Introduction

I decided to go and re-visit an old project.  Getting my Raspberry Pi to switch some old RF plugs.  I tried earlier this year or maybe last year and could never work it out.

This was prompted by a random google to this site.  https://www.instructables.com/id/Control-RF-Outlets-With-Raspberry-Pi-3-433mHz-Rece/  and as I had all the “bits” I thought I would try again.

Installing Hassbian on the Pi

Things are never simple.  Firstly getting a version of Hassbian onto the memory card seems to be much harder than I expected.  The link in the article above was not to the latest version (Why would it be?) and looking further I found the install instructions here. https://www.home-assistant.io/docs/installation/hassbian/installation/.  Once I had booted the Pi it seems that Hassbian was not corrected loaded so I upgraded in installation using  “sudo hassbian-config upgrade hassbian”.  This took a while to run (30+ mins). Then followed instructions and added

  1. Samba
    sudo apt-get update
    sudo apt-get upgrade
    sudo hassbian-config install samba
  2. WiringPi
    sudo git clone git://git.drogon.net/wiringPi
    cd wiring Pi
    sudo ./build
  3. Check using
    gpio -v
  4. Install RFSniffer
    sudo git clone git://github.com/timleland/rfoutlet.git /var/www/rfoutlet
    sudo chown root.root /var/www/rfoutlet/codesend
    sudo chmod 4755 /var/www/rfoutlet/codesend

Pi Hardware – Receiver and Transmitter

Lets have a look at the hardware. I have a breakout board from the Pi which helps.

Breakout board showing a very rough experiment
Shows the wiring a bit better

Transmitter
Transmitter

The hardware is wired like this
Receiver
– VCC – 5v (Black wire)
– GND – GND bottom left (white wire)
– DATA – P17 (grey wire)

Transmitter
– VCC – 5v (Red wire)
– GND – GND middle (green wire)
– DATA – P21 (blue wire)

Now all wired up and software running.

Nothing works…. yet…. the investigation

Time to run RFSniffer
sudo /var/www/rfoutlet/RFSniffer
and as expected there is a blank screen.  However clicking the BBSB remote seems to detect nothing even though the switch is turning on and off.

Well it seems I have not got what I want!  I wondered if the GPIO was receiving anything.  I found out I could install piscope and send the output to an X11 server.  http://abyz.me.uk/rpi/pigpio/piscope.html.  Some more installing of software Xming for the PC and pigpiod etc and I could then run piscope &.  What I discovered was a Channel 21 was indeed receiving data but masses of noise so anything I might have been sending looked like it was lost in a sea and so RFSniffer was not getting anything different.

Some more googling and went here http://abyz.me.uk/rpi/pigpio/examples.html#pdif2__433D. Downloaded the 433Mhz Keyfob RX/TX software.

Getting this running indicated that the receiver was indeed getting data but that the format of the data was not Manchester encoding which is what this software is meant to do.  

Getting the Bye Bye Standbye switches working

How do I find out what encoding does the BBSB use?  Google again!  It seems that BBSB is simple I found references to Home Easy Simple protocols.  I tried using Pilight but could not work out how to configure it easily so tried looking further.  Then found reference to KAKU OOK protocol.  I eventually landed up installing rfctl from here https://github.com/troglobit/rfctl.  The instructions are not for a newbie like me the command 

sudo apt install raspberry-kernel-headers

was OK but then when I looked in /lib/modules there seems to be 2 directories.  Which one?  I eventually used 14.14.70+ and not 14.14.70-v7+.  Took me a while to work out that I needed to do the following

cd /lib/modules/14.14.70+
sudo git clone https://github.com/troglobit/rfctl.git

but once past that simple bit the instructions are clear (but I needed to run as sudo)

cd rfctl/kernel
sudo make
sudo make install

cd rfctl/src
sudo make
cd make install

I have my BBSB set to A3 so trying the following worked a treat

this turns on

rfctl -p NEXA -g A -c 3 -l 1

and this turns off

rfctl -p NEXA -g A -c 3 -l 0

Now how to integrate this into Home Assistant.

When I Googled the remote model number WST-8500 and found the receiver model numbers are YCR-300,YCR-3500,WSR-1000,LCMR-1000,LCMR-300.  Not sure if this is going to help in the future or not.

Integrating into a single command

The next step is to be able to run this command remotely from the main Home Assistant server as I will not be using the HA on the Pi and will stop Hass from running at boot time.

So for passwordless ssh from the HA server to the Pi I created some ssh keys using command

ssh-keygen

on the HA server while logged in as the homeassistant user. (it turns out that HA is running under root user and so the .ssh keys from root could make this work but instead made HA server run under the homeasssistant user).

Next step create a new user on the Pi, gave the user sudo privileges (sudo user mod -aG sudo <username> and copied the keys to the Pi new user

cat ~/.ssh/id_rsa.pub | ssh <USERNAME>@<IP-ADDRESS> 'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys'

next added an entry to the etc/sudoers.d directory for the new user giving access to /usr/local/bin/rfctl, dev/rfctl, and /home/<new user>.

Following this I created a simple shell script to allow running of the command for the switches in position A3 and A4 (I have only 2). The script is given execution privileges
chmod a+x onff4.sh so it can be run using

./onoff4.sh

I can now run the script from the HA command line logged in a the home assistant user like this

ssh -t <newuser>@<piaddress> ./onoff4.sh

OMV – WordPress and NGINX – a further update

I was struggling with WordPress and permalinks.  The prior onfiguration I had developed worked fine until I switched to permalinks and then it seems to be picked up the js css etc from the root and not the alias.

Google is wonderful as I found this site – http://www.netz39.de/2014/installation-of-wordpress-in-a-subdirectory-with-nginx/.

I thought it was something to do with the way the try_files was working in relation to fastcgi.  It was.

So the wordpress config was update to be as below.  Note the location rewrite needs to be at the server level.

 

location @wordpress_rewrite {
rewrite ^/wordpress/(.*)$ /wordpress/index.php?$1;
}

location /wordpress {
alias /media/13eaa8c9-07ee-4dfc-9d1b-adfae0f0248d/www/wordpress;

add_header Content-Security-Policy "default-src 'self' https: data: 'unsafe-inline' 'unsafe-eval';" always;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
add_header X-Robots-Tag none;
add_header X-Download-Options noopen;
add_header X-Permitted-Cross-Domain-Policies none;

index index.php;
try_files $uri $uri/ @wordpress_rewrite;

location ~* /wordpress/(?:uploads|files)/.*\.php$ {
deny all;
}

location ~ ^/wordpress/(.+\.php)$ {
alias                           /media/13eaa8c9-07ee-4dfc-9d1b-adfae0f0248d/www/wordpress/$1;
fastcgi_split_path_info         ^(.+\.php)(/.*)$;
include fastcgi.conf;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $request_filename;
fastcgi_pass unix:/var/run/fpm-19e311b1-25e1-4257-8dbc-d49c5ec59015.sock;
}

location = /wordpress/favicon.ico {
access_log off;
log_not_found off;
}

location = /wordpress/robots.txt {
allow all;
access_log off;
log_not_found off;
}

location ~ /wordpress/\. {
deny all;
}

location ~* /wordpress/(?:config)/.*\.php$ {
deny all;
}

location ~* /wordpress/\.(js|css|png|jpg|jpeg|gif|ico)$ {
expires max;
log_not_found off;
}
}

Owntracks and Mosquitto

Came across Owntracks and decided that it would be fun to have my own Mosquitto repository.

Followed instructions here ….
https://www.digitalocean.com/community/questions/how-to-setup-a-mosquitto-mqtt-server-and-receive-data-from-owntracks

They work fine with the exception of needing to install mosquitto-clients to get the mosquitto_sub command working and leaving off the ip address in the mosquitto.conf file from the listener entry. Looking at the log later on the connection is from the ip address of the router on the local lan for external connections and that probably was the address needed to get things working if an ip address was used.

So when I tested and changed the location and config items multiple items are written to the mosquitto database. This then looks bad in the recorder as there are loads of entries which don’t look pretty!
These are in var / spool / owntracks / recorder / store and then either last or rec. If you remove all entries except the ones you want from both then things look pretty again.

It is also worth installing mqtt-spy from mqhive – http://kamilfb.github.io/mqtt-spy/ and publishing a null record without any data or retain so that the mosquitto database does not have then entries any more.

the mosquitto database is normally in var/lib/mosquitto but the location can be changed in the mosquitto.conf or conf.d file in etc/mosquitto.

Now it got tricker when I wanted to publish the steps from my iPhone. Publishing a command did nothing.  I found this comment on GitHub …. ”

Remote commands need to be enabled with remoteConfig: export your configuration, add cmd: true, and re-import (e.g. by sending the .otrc file to iOS and open in OwnTracks).

Once you’ve done that, you can send remote commands. Note though, that the iOS app is put to sleep by the OS which means that when it’s in the background you will have to wait until it wakes up for the command to take effect / be processed; this can take a while. Hence we suggest you post commands to iOS with QoS=2″

So I did this and all is well.

mosquitto_pub -q 2 -h xxxx -p 8883 -t owntracks/owntracks/michael-iPhone6s/cmd -u owntracks -P xxx -m ‘{“_type” : “cmd”, “action”: “reportSteps”}’

and this opened the BBC new in the Featured tab.  Cool….

mosquitto_pub -q 2 -h xxx -p 8883 -t owntracks/owntracks/michael-iPhone6s/cmd -u owntracks -P xxx -m ‘{“action”: “action”, “url”: “http://news.bbc.co.uk”, “_type”: “cmd”}’

I was a bit puzzled about why the topic was owntracks/owntracks but on further reading of the documentation the default topic is owntracks/username/devicename and as I was using a username of owntracks the second pat of the topic derived from that.

Re-naming a table in MyWebSQL

So this was trickier than I thought.

I have a .sql dump of a database and when I imported it it was saved as wordpressblog.

I wanted it to be naylorfamily.

There is no rename function and so I needed to revert back to the command line and did it like this

mysql -u xxxxx -p"xxxxxx" wordpress -sNe 'show tables' | while read table; do mysql -u xxxxx -p"xxxxxx" -sNe "RENAME TABLE wordpress.$table TO naylorfamily.$table"; done

More Firefly III on OMV

I decided to have another go installing the latest version using PHP 7.2.

The instructions are pretty clear on the Firefly website https://firefly-iii.readthedocs.io/en/latest/installation/upgrading.html but I still encountered some problems.

  1. Installing PHP.7.2 works fine but I got an error running
    php artisan migrate --env=production

    it turned out I did not have php7.2-mysqld installed.  Installing this fixed the error.
  2. I then had to sort out running NGINX with PHP7.2.  It turned out to be easier than I expected.  I added a new pool file into /etc/php/7.2/pool.d instead of /etc/php/7.0/pool.d and changed the .conf file for firefly in /etc/NGINX/xxx.d/ file created when php-fpm was restarted (systemctl restart php7.2-fpm).
  3. Permissions problems everywhere.  While www-data should have read and write access it also needs create access in all the folders a the root install.  In the end the only way I successfully got this working was to used the Shared Folders functionailty in OMV.  I am not sure why the recursive chmod did not work but it seems the ability to have write access to new folders created by firefly did not work.