yfinance 0.2.3 on raspberry pi

I have a silly little script that checks the shares each day and sends me a telegram, but it stopped working. After an hour or so of analysis I determined that the version of yfinance on raspberry pi that I had, 1.74, was returning None on any call. If I upgrade to 2.3 on Windows (update pip,pip install -upgrade pip, then pip install –upgrade yfinance, it all starts working again.

Getting it to work on Raspberry Pi was not so simple. This is a record of what I did.

Show the currently installed packages in the venv

source ./.venv/bin/activate
pip list

Upgrade pip and try to upgrade yfinance

pip install --upgrade pip
pip install --upgrade yfinance

You get an error saying that rust is the wrong version. Doing sudo apt install rustc does not install a version of rust that is high enough. So you turn to rust istelf and use:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh 

Then follow the instructions. You need to kill the shell or read the message and source “$HOME/.cargo/env”. Then try to upgrade yfinance again.

pip install --upgrade yfinance

It now fails with:

error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for cryptography
Failed to build cryptography
ERROR: Could not build wheels for cryptography, which is required to install pyproject.toml-based projects

So try to get gcc working. I read that installing gcc libffi-dev libssl-dev and python3-dev might help so did that, though I already had gcc and python3-dev installed so just needed this:

sudo apt-get install libffi-dev libssl-dev
pip install --upgrade yfinance

And THAT fixed it 😁so I now have yfinance 0.2.3 and yfinance works.

arduino as mqtt temperature sensor

My office gets pretty cold and I had worried that the heater would come on if it went below 5°C. This was until I went in there and discovered that the thermostat from Honeywell will not read lower than 5°C!

Now I have grafana I was wondering how to get a reading of the temperature in there. I have lot of kit kicking around from my arduino days so I thought I would give it a whirl. The aim was for the arduino to push the current temperature to the mosquitto broker on openhabian then read it out to influxdb and eventually to grafana.

I ended up using:

  • arduino uno
  • arduino uno ethernet shield
  • DS18B20 temperature sensor
  • 4.7 kΩ resistor
  • breadboard and leads

Recipe

I used this to remind me how to get the temperature sensor working: https://create.arduino.cc/projecthub/TheGadgetBoy/ds18b20-digital-temperature-sensor-and-arduino-9cc806. Then with a bit of searching and playing around I got the whole kaboodle working. There was a lot of intelligent copy and paste and I have no idea how much of this is redundant but it bloody works (code below).

Weirdly, the graph sparklines did not work in the temperature display in grafana. https://community.grafana.com/t/sparkline-does-not-appear-on-single-stat-graph/566 helped me work out that I needed to do a GROUPBY in the query (I used 15m). You can see the result at the top of the page.

/********************************************************************/
// First we include the libraries
// Temp Sensor
#include <OneWire.h> 
#include <DallasTemperature.h>
// Ethernet Shield
#include <SPI.h>
#include <Ethernet.h>
// Mqtt
#include <ArduinoMqttClient.h>
/********************************************************************/
// Data wire is plugged into pin 2 on the Arduino 
#define ONE_WIRE_BUS 2 
/********************************************************************/
// Setup a oneWire instance to communicate with any OneWire devices  
// (not just Maxim/Dallas temperature ICs) 
OneWire oneWire(ONE_WIRE_BUS); 
/********************************************************************/
// Pass our oneWire reference to Dallas Temperature. 
DallasTemperature sensors(&oneWire);
/********************************************************************/ 

// Enter a MAC address for your controller below.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

// Set the static IP address to use if the DHCP fails to assign
IPAddress ip(192, 168, 0, 23);
IPAddress myDns(192, 168, 0, 1);

EthernetClient net;

MqttClient mqttClient(net);

const char broker[] = "openhabian";
int        port     = 1883;
const char topic[]  = "arduino/temperature/";

char clientId[] = "ToM_arduino_001";
char username[] = "openhabian";
char password[] = "averybigsecret";

const long interval = 1000;
unsigned long previousMillis = 0;


void setup(void) 
{ 
 // start serial port 
 Serial.begin(9600); 
 Serial.println("Dallas Temperature IC Control Library Demo"); 
 // Start up the library 
 sensors.begin(); 
  // start the Ethernet connection:
  Serial.println("Initialize Ethernet with DHCP:");
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // Check for Ethernet hardware present
    if (Ethernet.hardwareStatus() == EthernetNoHardware) {
      Serial.println("Ethernet shield was not found.  Sorry, can't run without hardware. :(");
      while (true) {
        delay(1); // do nothing, no point running without Ethernet hardware
      }
    }
    if (Ethernet.linkStatus() == LinkOFF) {
      Serial.println("Ethernet cable is not connected.");
    }
    // try to configure using IP address instead of DHCP:
    Ethernet.begin(mac, ip, myDns);
  } else {
    Serial.print("  DHCP assigned IP ");
    Serial.println(Ethernet.localIP());
  }
  // give the Ethernet shield a second to initialize:
  delay(1000);

  Serial.println("You're connected to the network");
  Serial.println();

  // You can provide a unique client ID, if not set the library uses Arduino-millis()
  // Each client must have a unique client ID
  mqttClient.setId(clientId);

  // You can provide a username and password for authentication
  mqttClient.setUsernamePassword(username, password);

  Serial.print("Attempting to connect to the MQTT broker: ");
  Serial.println(broker);

  if (!mqttClient.connect(broker, port)) {
    Serial.print("MQTT connection failed! Error code = ");
    Serial.println(mqttClient.connectError());

    while (1);
  }

  Serial.println("You're connected to the MQTT broker!");
  Serial.println();
} 
void loop(void) 
{ 
 // call sensors.requestTemperatures() to issue a global temperature 
 // request to all devices on the bus 
/********************************************************************/
 Serial.print(" Requesting temperatures..."); 
 sensors.requestTemperatures(); // Send the command to get temperature readings 
 Serial.println("DONE"); 
/********************************************************************/
 Serial.print("Temperature is: "); 
 Serial.print(sensors.getTempCByIndex(0)); // Why "byIndex"?  
   // You can have more than one DS18B20 on the same bus.  
   // 0 refers to the first IC on the wire 
   delay(10000); 
   // call poll() regularly to allow the library to send MQTT keep alives which
  // avoids being disconnected by the broker
  mqttClient.poll();

  // to avoid having delays in loop, we'll use the strategy from BlinkWithoutDelay
  // see: File -> Examples -> 02.Digital -> BlinkWithoutDelay for more info
  unsigned long currentMillis = millis();
  
  if (currentMillis - previousMillis >= interval) {
    // save the last time a message was sent
    previousMillis = currentMillis;

    Serial.print("Sending message to topic: ");
    Serial.println(topic);


    // send message, the Print interface can be used to set the message contents
    mqttClient.beginMessage(topic);
    mqttClient.print(sensors.getTempCByIndex(0));
    mqttClient.endMessage();

    Serial.println();

  }
} 

influxdb and grafana

grafana

I have long wanted historical data on temperatures. Not sure why, but it might be interesting. Enter the influxdb persistence plugin and grafana (now I need to work out how to back them up too I guess).

Backing up influxdb turns out to be easy and I will add this to the backup script:

influxd backup -portable -database openhabian /mnt/nas/backup/openhabian/influxdb

rpi can just about run inlfuxdb 1.6 and can be installed by apt, and was enough. I just followed the instructions below then set it up in OpenHAB. I then had to work out how to actually store information in teh database. grafana was much easier.

Influxdb notes

  • https://docs.influxdata.com/influxdb/v1.6/introduction/installation/ but I used apt then followed the configuration instructions
  • You need to go in to the CLI and create a database, user and password which are then configured in openhab.
  • Then edit /etc/openhab/persistence/influxdb.persist – not sure about the line “Temperature* : strategy = everyHalfHour, everyChange”
sudo vi /etc/openhab/persistence/influxdb.persist 

influxdb.persist

Strategies {
    everyMinute : "0 * * * * ?"
    everyHour   : "0 0 * * * ?"
    everyHalfHour       : "0 0,30 * * * ?"
    everyDay    : "0 0 0 * * ?"

    default = everyChange
}

Items {
    Tasmota4Fridge_Tas04PowerNow  : strategy = everyMinute
    Tasmota2_Watts  : strategy = everyMinute
    EMONPIMQTTThing_HousePowerwatts : strategy = everyMinute
    KitchenEvohome_Temperature : strategy = everyHalfHour, everyChange
    MasterBedEvohome_Temperature : strategy = everyHalfHour, everyChange
    Tasmota01PowerNowW : strategy = everyMinute, everyChange
    Tasmota3TVStack_Tasmota03PowerNow : strategy = everyMinute
    WeatherandForecast_OutdoorTemperature : strategy = everyHalfHour, everyChange
    emonpiMQTTThing_EmonPiExternelTemp : strategy = everyHalfHour, everyChange
    SnugThermostat_Temperature : strategy = everyHalfHour, everyChange
    Temperature* : strategy = everyHalfHour, everyChange

}

grafana notes

I just did all this then set up influxdb and created some graphs:

https://grafana.com/tutorials/install-grafana-on-raspberry-pi/

There is no obvious backup solution yet, but I will explore this: https://github.com/ysde/grafana-backup-tool

Backing up Openhabian

I am using OpenHab more and more and not having a backup was starting to worry me. There are better solutions, but this works for me. I have a 1TB usb PCIe HDD which the OpenHabian is now running as the house nas, and this solution makes a backup each night and copies the newest file to the NAS with a full backup each Sunday.

It has to be run as root cron:

sudo crontab -e -u root

Then this is the cron for root:

10 2 * * * /home/openhabian/bin/backup-openhabian.sh

And this is the script:

#!/bin/bash

##############################
#  MUST be run by root cron!!
##############################
if [[ $(date +%u) -gt 6 ]]; then
    openhab-cli backup --full
else
    openhab-cli backup
fi

# Copy to the nas
echo Copying the newset file to the NAS folder...
file=$( ls -t /var/lib/openhab/backups/* | head -1 )
cp $file /mnt/nas/backup/openhabian/

# delete files older than 7 days
echo Deleting anything older than 7 days...
find /var/lib/openhab/backups/* -mtime +7 -exec rm {} \;
echo Done!

Restoring

https://community.openhab.org/t/solved-backup/57666 is for OH2, but I guess it will work the same. I really need to practise that with a blank SD card.

sudo systemclt stop openhab
openhab-cli restore /var/lib/openhab/backups
sudo systemclt start openhab