... ...

Anagram Solver Solar Artwork

TuftedPuffin.co.uk

... ...

 

... ...
 

Live Data

Live

Today

Yesterday

Energy

PV

Electricity Usage

Write Up

Introduction

OneWire

Scripts

Mysql

Gnuplot

Energy

Theoretical

SVG

Behaviour

PV Monitor

Current Monitor

Bandwidth Monitor

Monitoring Scripts

A number of bash scripts are used to acquire and database the data. Other scripts are called on a regular basis to update this and other websites

The main monitoring script is :-


#!/bin/sh

place="/home/steve/monitor"

cd $place

while [ true ]
do
sleep 30

$place/onewireall.sh
ncftpput -u tuftedpuffin.co.uk -p XXXXXX -a ftp.tuftedpuffin.co.uk htdocs/House $place/onewire.out
mysql -usteve house <$place/onewire.sql
done

As can be seen, monitor.sh is in an infinite loop, with a 30 second daily. It will call onewireall.sh, which does the bulk of the work and then ftp the current readings, in onewire.out, up to the website as well as invoking the generated sql script to append the latest readings to the database..

The onewireall.sh script is as follows:-


#!/bin/bash

place="/home/steve/monitor"

cd $place

sqlfile=$place/onewire.sql
outfile=$place/onewire.out


rm $sqlfile
rm $outfile

echo 'INSERT INTO HouseTemperature (' >$sqlfile
echo 'date,' >>$sqlfile
echo 'Yellow1Study ,' >>$sqlfile
echo 'Yellow2Outside ,' >>$sqlfile
echo 'Yellow2OutsideHumidity ,' >>$sqlfile
echo 'Yellow3Larder ,' >>$sqlfile
echo 'Yellow4WorkshopLight ,' >>$sqlfile
echo 'CollectorReturn ,' >>$sqlfile
echo 'CollectorFeed ,' >>$sqlfile
echo 'Chip2 ,' >>$sqlfile
echo 'Chip3 ,' >>$sqlfile
echo 'WorkshopRoofSpace ,' >>$sqlfile
echo 'ColdWaterTank ,' >>$sqlfile
echo 'MainBedroom ,' >>$sqlfile
echo 'Loft1 ,' >>$sqlfile
echo 'Lounge ,' >>$sqlfile
echo 'SolarOut ,' >>$sqlfile
echo 'SolarIn ,' >>$sqlfile
echo 'MidTank ,' >>$sqlfile
echo 'TwixtSolar ,' >>$sqlfile
echo 'TopOfTank ,' >>$sqlfile
echo 'GasIn ,' >>$sqlfile
echo 'TankBottom ,' >>$sqlfile
echo 'TwixtGas ,' >>$sqlfile
echo 'GasOut ,' >>$sqlfile
echo 'ColdWaterIn ,' >>$sqlfile
echo 'HotWaterOut , ' >>$sqlfile
echo 'OutsideSun ,' >>$sqlfile
echo 'OutsideSunV ,' >>$sqlfile
echo 'SolarPipe1, ' >>$sqlfile
echo 'SolarPipe2 ' >>$sqlfile
echo ')' >>$sqlfile
echo 'VALUES' >>$sqlfile
echo '(' >>$sqlfile

date -u +\"%Y%m%d%H%M%S\" >>$sqlfile
echo ',' >>$sqlfile
echo -n "TIME|" >$outfile
date >>$outfile

outputValue() {
# param $1 - OneWire directory path and file name
# param $2 - Tag name for SVG
# param $3 - Set if last thing in list

t=cat $1`
if [ $? != 0 ]
then
t='-10000
echo $1 failed
fi

# output sql values
echo -n $t >>$sqlfile
if [ -z $3 ]
then
echo ',' >>$sqlfile
fi
# output SVG values
echo -n "|"$2"|" >>$outfile
echo -n $t >>$outfile
# output CSV values
echo -n $t >>$place/onewire.csv
if [ -z $3 ]
then
echo -n ',' >>$place/onewire.csv
fi
}

outputValue /var/1wire/Yellow1Study/temperature "TStudy1"
outputValue /var/1wire/Yellow2OutsideHumidity/temperature "Outside"
outputValue /var/1wire/Yellow2OutsideHumidity/HIH4000/humidity "HOutside"
outputValue /var/1wire/Yellow3Larder/temperature "TLarder"
outputValue /var/1wire/Yellow4WorkshopLight/temperature "TWorkshop"
outputValue /var/1wire/CollectorReturn/temperature "CollectorReturn"
outputValue /var/1wire/CollectorFeed/temperature "CollectorFeed"
outputValue /var/1wire/Chip2/temperature "TChip2"
outputValue /var/1wire/Chip3/temperature "TChip3"
outputValue /var/1wire/WorkshopRoofSpace/temperature "TWorkshopRoofSpace"
outputValue /var/1wire/ColdWaterTank/temperature "ColdWaterTank"
outputValue /var/1wire/MainBedroom/temperature "MainBedroom"
outputValue /var/1wire/Loft1/temperature "Loft"
outputValue /var/1wire/Lounge/temperature "TLounge"
outputValue /var/1wire/SolarOut/temperature "SolarOut"
outputValue /var/1wire/SolarIn/temperature "SolarIn"
outputValue /var/1wire/MidTank/temperature "MidTank"
outputValue /var/1wire/TwixtSolar/temperature "TwixtSolar"
outputValue /var/1wire/TopOfTank/temperature "TopOfTank"
outputValue /var/1wire/GasIn/temperature "GasIn"
outputValue /var/1wire/TankBottom/temperature "TankBottom"
outputValue /var/1wire/TwixtGas/temperature "TwixtGas"
outputValue /var/1wire/GasOut/temperature "GasOut"
outputValue /var/1wire/ColdWaterIn/temperature "ColdWaterIn"
outputValue /var/1wire/HotWaterOut/temperature "HotWaterOut"
outputValue /var/1wire/OutsideSun/temperature "TOutsideSun"
outputValue /var/1wire/OutsideSun/VAD "VOutsideSun"
outputValue /var/1wire/SolarPipe1/temperature "SolarPipe1"
outputValue /var/1wire/SolarPipe2/temperature "SolarPipe2" "LAST"

echo -n "|LWorkshop|" >>$outfile
cat /var/1wire/Yellow4WorkshopLight/HIH4000/humidity >>$outfile

while read line
do
echo $line
t=$line
done <$place/sumEnergy.out
echo -n "|SolarEnergy|"$t >>$outfile

while read line
do
echo $line
t=$line
done <$place/sumGasEnergy.out
echo -n "|GasEnergy|"$t >>$outfile

date +%Y%m%d%H%M%S >>$place/onewire.csv

echo ');' >>$sqlfile


This script generates the 3 output formats that I wanted, an sql script to add the data to the database, a file that can be read by the SVG monitor website and it also appends to a CSV file.

There are 3 parts to onewireall.sh script, the first part simply generates the start of the SQL INSERT statement by echoing the column names to the sqlfile. The second part reads the values from the OWFS and outputs them into the various required formats. This is done by calling the function "outputValue" with the directory path for the reading and the tag name that is expected by the SVG. The values are read by assigning the output of "cat" to a shell variable. The return status from "cat" is checked so that if the file does not exist or if some other error occurs, an error value can be substituded. The third part of the file deals with a few oddities in my setup. I don't want to database the workshop light status, so this is treated seperately, also extracting the total energy generated requires a little bit of special scripting.

An example of the sql comamnd generated looks like this :-


INSERT INTO HouseTemperature (
date,
Yellow1Study ,
Yellow2Outside ,
Yellow2OutsideHumidity ,
Yellow3Larder ,
Yellow4WorkshopLight ,
CollectorReturn ,
CollectorFeed ,
Chip2 ,
Chip3 ,
WorkshopRoofSpace ,
ColdWaterTank ,
MainBedroom ,
Loft1 ,
Lounge ,
SolarOut ,
SolarIn ,
MidTank ,
TwixtSolar ,
TopOfTank ,
GasIn ,
TankBottom ,
TwixtGas ,
GasOut ,
ColdWaterIn ,
HotWaterOut ,
OutsideSun ,
OutsideSunV ,
SolarPipe1,
SolarPipe2
)
VALUES

(
"20110402165234"
,
18.9688,
12.0938,
83.1202,
18.875,
19.25,
27,
40.5,
-10000,
19.625,
19.875,
16.25,
18.5,
18.8125,
19.4375,
27.3125,
38.875,
33.8125,
31.4375,
44.1875,
38.875,
25.75,
39.5,
46.0625,
29.0625,
45.125,
11.75,
6.57,
30.4375,
27.375

The SVG tag file is a list of names and values seperated by a delimeter, I use the vertical bar, it is relatively easy to read this data and match the values to SVG items, see the section on SVG display later.

TIME|Sat Apr 2 17:52:34 BST 2011
|TStudy1|18.9688|Outside|12.0938|HOutside|83.1202|TLarder|18.875|TWorkshop|19.25|CollectorReturn|27|CollectorFeed|40.5|TChip2|-10000|TChip3|19.625|TWorkshopRoofSpace|19.875|ColdWaterTank|16.25|MainBedroom|18.5|Loft|18.8125|TLounge|19.4375|SolarOut|27.3125|SolarIn|38.875|MidTank|33.8125|TwixtSolar|31.4375|TopOfTank|44.1875|GasIn|38.875|TankBottom|25.75|TwixtGas|39.5|GasOut|46.0625|ColdWaterIn|29.0625|HotWaterOut|45.125|TOutsideSun|11.75|VOutsideSun|6.57|SolarPipe1|30.4375|SolarPipe2|27.375|LWorkshop| 2.45793|SolarEnergy|16.4641143456101|GasEnergy|10.3138937465847

As stated earlier, if any of the OWFS files can not be read then a value of -10000 is substituted. This can easily be filtered out when the data is displayed.

Other scipts are run via CRON to extract data from the database and generate graphical plots of the data.

Hourly.sh is run once an hour, on the hour it generates the png files and uploads them to the website.


#!/bin/bash # Run hourly place="/home/steve/monitor" cd $place touch $place/doinghourly mysql -usteve house <$place/today.sql >$place/today.out mysql -usteve house <$place/energyToday.sql >$place/et.out $place/kwh 50.652 -1.172 45 155 >$place/sun.out rm $place/sqlEnergy.sql $place/smooth $place/et.out 5 17.5 >$place/smooth.out ncftpput -u tuftedpuffin.co.uk -p Old317ehho177s ftp.tuftedpuffin.co.uk htdocs/Solar $place/sumEnergy.out ncftpput -u lakecommon@talktalk.net -p Old317ehho177s www.lakecommon.talktalk.net House $place/sumEnergy.out plotAndSend() { # $1 is gnuplotcontrol file # $2 is name of output file from gnuplot gnuplot $1 >$2 ncftpput -u uuuuuuuuuuuu.co.uk -p xxxxxxxx ftp.xxxxxxxxxxxx.co.uk htdocs/Solar $2 ncftpput -u uuuuuuuuuuuu.net -p xxxxxxxx www.xxxxxxxxxxxxa.net House $2 cp $2 /var/www/House/ } plotAndSend $place/cylinderToday.p $place/ct.png plotAndSend $place/solarToday.p $place/st.png plotAndSend $place/houseToday.p $place/ht.png plotAndSend $place/gasToday.p $place/gt.png plotAndSend $place/energyToday.p $place/et.png plotAndSend $place/sunshineToday.p $place/lt.png plotAndSend $place/sun2.p $place/sun.png plotAndSend $place/sunele.p $place/sunele.png plotAndSend $place/sunazi.p $place/sunazi.png plotAndSend $place/sunstrength.p $place/sunstrength.png plotAndSend $place/sunpower.p $place/sunpower.png plotAndSend $place/illumToday.p $place/illumt.png $place/lightning.awk lightning.out plotAndSend $place/lightningDelta.p $place/lightningDelta.png plotAndSend $place/lightningToday.p $place/lightningt.png

The Hourly.sh script extracts data from the database using a set of sql scripts, to generate text dat that can then be read by gnuplot. It also invoked the utility kwh, this is a "C" program based on solpos (see later), which calculates the azimuth and elevation of the sun, and also calculates the theoretical watts per square metre that could be shining on the panels if it was a clear day. Gnuplot is invoked with various script files to generate various png files that chart the data. These png files are then ftp'd to the website. Note that I use a function "plotAndSend" to do most of the work, this keeps the script much neater than would otherwise be the case

Daily.sh is run once per day at midnight and generates more png files.


#!/bin/bash # Run hourly place="/home/steve/monitor" cd $place mysql -usteve house <$place/yesterday.sql >$place/yesterday.out mysql -usteve house <$place/energyYesterday.sql >$place/ey.out plotAndSend() { # $1 is gnuplotcontrol file # $2 is name of output file from gnuplot gnuplot $1 >$2 ncftpput -u uuuuuuuuuuuu.co.uk -p xxxxxxx ftp.xxxxxxx.co.uk htdocs/Solar $2 ncftpput -u uuuuuuuuuuuu.net -p xxxxxxx www.xxxxxxx.net House $2 cp $2 /var/www/House/ } plotAndSend $place/cylinderYesterday.p $place/cy.png plotAndSend $place/solarYesterday.p $place/sy.png plotAndSend $place/houseYesterday.p $place/hy.png plotAndSend $place/gasYesterday.p $place/gy.png plotAndSend $place/energyYesterday.p $place/ey.png plotAndSend $place/sunshineYesterday.p $place/ly.png plotAndSend $place/histo.p $place/histo.png

Evening.sh is run once per day at 23:30 and calculates the energy added by Solar and Gas into the cylinder.


#!/bin/bash
# Run daily

place="/home/steve/monitor"

cd $place

rm $place/sqlEnergy.sql
$place/smooth et.out 5 17.5 >$place/smooth.out
mysql -usteve house <$place/sqlEnergy.sql
mysql -usteve house <$place/sumEnergy.sql >$place/sumEnergy.out
mysql -usteve house <$place/sumGasEnergy.sql >$place/sumGasEnergy.out

For all of these scripts it is important to remember that CRON will run a command from the users home directory, it is therefore essential to make sure that explicit paths are used in all of the scripts, hence the use of $place and aa cd $place to ensure that the script is running in the correct directory.

SQL

This page is best viewed using Firefox Spread Firefox Affiliate Button

 

... ...