Add presentation. Fix docs, docker-compose and bandswitcher

Readme.md

@@ -1,20 +1,18 @@
 # what is it?
 
-![alt text](screenshot.png "Logo Title Text 1")
-
 * a set of tools to export your personal WSJT-X
   * live reception data into prometheus or mysql
   * historical reception data (since 2019) into mysql
 * a tool to poll pskreporter and export your personal 
-  * live transmission data to prometheus or mysql 
+  * live transmission data into prometheus or mysql 
 
 every time your wsjtx is/was running you are/were generating nice data points, which can be queried using grafana.
 this can be very interesting to evaluate your **personal** hf conditions from the current and the past.
 
 some use cases: 
 * choose dxcc, band, mode and timeframe in all combinations so see when your were successfully able to receive and transmit, to plan future activities
-* compare input of different station like for antenna comparions
+* compare data of different station for antenna comparions etc.
-* have nice wallboard for contesting or dxpedition
+* have a nice and useful wallboard for contesting or dxpedition
 * display rx/tx data on a map with custom timespan and filters
 * compare hf propagration forecasts to your personal situation
 * use grafana alerts or prometheus alertmanager to send yourself alerts when your stations receives something interesting
@@ -28,7 +26,7 @@ have fun!
 
 ## prerequisites
 
-* ALL.TXT with interesting data and/or running hamradio station to provide live data
+* one or more ALL.TXT with interesting data and/or running hamradio station to provide live data
 * grafana
 * prometheus or mysql
 
@@ -40,8 +38,6 @@ have fun!
 * **wsjtx-exporter**
   * follows live traffice in ALL.txt
   * supports prometheus and mysql
-* **alltxt2csv**
-  * reads whole ALL.txt since 2019 and creates csv-file which can be imported into mysql
 * **misc**
   * sample dashboards
   * sql table definitions
@@ -58,6 +54,7 @@ pro
 * you get nicer graphs with counters and bars if you ask me
 * you can use alertmanager if you want advanced alerting
 * optimized for timeseries data
+
 con
 * not designed for keeping long timeseries (weeks or some months are fine)
 * not possible to import historical data
@@ -69,10 +66,11 @@ pro:
 * you can import your ALL.txt since 2019
 * you can store a lot of data over a long timeframe
 * you can propably build more complex queries in sql
+
 con:
 * in comparison to prometheus not optimized for timeseries, but still good enough
 * gauge based graphs are not as nice as counter/bars if you ask me
-* you can use *only* graphanas internal alerting
+* you can use *only* grafanas internal alerting (or do something else)
 
 both allow distributed setups with multiple wsjtx instances submitting their data to a central prometheus or mysql service.
 you can as well run both in parallel and use prometheus for a live overview and mysql for historical evaluations.
@@ -95,7 +93,7 @@ thats quite normal as there are propably a lot of strange/broken messages in you
 
 ### does it work with wsjt-x 2.3.0.RC2?
 
-no, there is a bug which prevents the ALL.TXT to contain them timestamp, but it should be fixed in the next release.
+no, there is a bug which prevents the ALL.TXT to contain the timestamp, but it should be fixed in the next release.
 
 ### what is the state of this?
 
@@ -106,7 +104,7 @@ feedback and pull requests are highly appreciated!
 here be dragons:
 * callsign lookup performance done with regex in one goroutine is unneccessary slow for alltxt2csv
 * grids for ALL.TXT entries are always looked up by dxcc instead if being tracked in the file
-* there is a lot of space for improvements for the dashboards
+* there is a lot of room for improvements in the dashboards
 
 ### grafana says too many rows for mysql datasource
 
@@ -159,5 +157,7 @@ alerting
 
 * WSJT-X, Grafana, Prometheus, MySQL Teams
 * Philip Gladstone for pskreporter.info
-* Todd Neal     for his nice work on https://github.com/tzneal/ham-go
+* Todd Neal for https://github.com/tzneal/ham-go
-* Stefan Prodan for his nice work on https://github.com/stefanprodan/dockprom
+* Stefan Prodan for https://github.com/stefanprodan/dockprom
+* Markus Lindenberg DO8ML for helping with a lot of stuff
+* Sebastian Schaumann DL7LE for testing and a lot of good ideas and questions

devnotes.md

@@ -1,33 +1,37 @@
 # assumptions/ideas
 
+* parsing ALL.txt is better than udp - udp cant be rewind
 * keep client as stupid/uncomplicated as possible
+  * no decentral maintenance should be needed if something changed
   * noch parsing/resolving, just postition finding
 
 # before release
 
-* multiple repos?
-  * alltxt2http & http-exporter (inkl. shared/httpstuff und shared/wsjtx)
-  * pskreporter-exporter
-  * dashboards n docs
-
 * alltxt2csv kann weg oder?
   * resolving lokal nur bedingt sinnvoll
     * profitiert nicht von ggf. vorhandenen grids im cache
   * wenn station und identifier bekannt sind, kann das auch der admin tun
+    * und das kann dann auch der wsjtx-exporter inkl. cache lookups etc. uebernehmen 
 
 * think about:
   * logische trennung in zeilenverarbeitung und 'spotting' (spot in mysql&prometheus exportieren)
     * take line -> parse line -> resolve grid and lotw for line ?
 
+* stationpi
+  * wsjtx works
+  * alltxt2http works
+  * wireguard works
+  * audio via pulse-tcp seems to work..
+  * write doc!
+
 * http-exporter
+  * callsignlabel aus prometheus entfernen?????
   * errorhandling
     * db down
       * who keeps track of the missing lines???
     * lookup error (not an error)
-  * add instance field to mysql
-    * what happes if no instance is provided?
   * improve logging
-  * counter vs histogram in prometheus?
+  * counter vs histogram in prometheus? (fuer signal)
   * http 200 OK synchron warten lassen bis alles fertig ist?
     * scheint tendenziell eine gute idee zu sein.. wenn db problem hat, kann client aufhoeren..
 
@@ -39,10 +43,11 @@
 
 * alltxt2http
   * what about readall/batchmode?
+  * errorhandling/logging
+    * see journal for obviously ending without any logs?!
 
 * systemd restart problematik (failed too fast oder soo ;)
 
-
 * Username=Station (=~ Callsign) right?!?
   * call und password in htacces und fertig!
   * auth. user kann seine instance selber waehlen
@@ -62,13 +67,13 @@
 * push images to dockerhub
 
 * doc
-  * german docs..
   * fix single tool readmes
   * hints for using grafana
     * intervals
     * fold row which are not needed
     * use filters!
   * server und/oder skript/readme zum aufsetzen
+  * german docs..
 
 # later
 

docker-compose/docker-compose.yml

@@ -48,6 +48,17 @@ services:
 #    labels:
 #      org.label-schema.group: "monitoring"
 
+  redis:
+    image: redis
+    container_name: cache
+    expose:
+      - 6379
+    restart: unless-stopped
+    networks:
+      - monitor-net
+    labels:
+      org.label-schema.group: "monitoring"
+
   grafana:
     image: grafana/grafana:7.3.4
     container_name: grafana
@@ -145,7 +156,7 @@ services:
     image: 10.0.73.1:5000/wsjtx-exporter:latest
     container_name: wsjtx-exporter
     ports: 
-      - "4000:4000"
+      - "4001:4000"
     environment:
       - PROMETHEUS=true
       - MYSQL=true

misc/bandswitcher.sh

@@ -1,20 +1,36 @@
 #!/bin/bash
 
-RIGCTL=$(which rigctl)
+# stay DELAY seconds on every band
-PARAMETERS="-m 2"
 DELAY="300"
 
-BANDS="14074000 21074000 28074000"
+# local rigctld, can be running dummy rig!
+PARAMETERS="-m 2"
+BANDS=(14074000 21074000 28074000)
+
+# enable gqrx intergration
+GQRX="1"
+GQRX_PARAMETERS="-m 2 -r localhost:7356"
+GQRX_BANDS=(139074000 146000740 153000740)
+
+# find rigctl binary
+RIGCTL=$(which rigctl)
 
 # wait for full minute
+echo "wait for full minute.."
 sleep $(((60 - $(date +%s) % 60)-1))
 
 # start switching
-while true
+let len=${#BANDS[*]}-1
-do
+while true ; do
-  for band in $BANDS
+  for i in $(seq 0 $len); do
-  do
+    date
-    $RIGCTL $PARAMETERS F $band
+    echo "switching local rigctl to ${BANDS[$i]}"
+    $RIGCTL $PARAMETERS F ${BANDS[$i]}
+
+    if [ "$GQRX" = "1" ]; then
+      echo "switching gqrx to ${GQRX_BANDS[$i]}"
+      $RIGCTL $GQRX_PARAMETERS F ${GQRX_BANDS[$i]}
+    fi
     sleep $DELAY
   done
 done

presentation.md

@@ -0,0 +1,47 @@
+# O49 Vorstellung
+
+* Funkwetterprognosen schwer verstaendlich..
+  * muss anschaulicher gehen - siehe Gridtracker
+  * WSJT-X generiert viele spannenden Daten (ab 2019)
+    * NICHT die Logdaten... sondern ALL.TXT :)
+
+* Idee - Daten in Grafanakompatible Datenquellen packen
+  * was ist Grafana?
+  * Datenmenge
+
+* Nur Daten von meinem WSJT-X sind nicht soooo interessant
+
+* PSKReporter
+  * aktuell alle 5 Minuten abolen und speichern
+    * bald mit Livedaten :)
+
+* Benefits/Features!
+  * Longtime Auswertung
+  * Wallboard
+  * DX-Jagd Vorbereitung
+  * Callsign Tracking
+  * Vergleich von Stationen
+
+* Know Issues
+  * know your data/query!
+    * live counter vs tx
+    * dl3sd 24/7 vs dl7le 6/7
+    * continental distribution
+    * map zeigt nur eine dimension (dxcc ODER band ODER receiver) -> keine ueberlagerung moeglich
+  * alles wip!
+    * viele features nicht drin
+    * Dashboards
+    * db
+    * komponenten
+  * Achtung Last!
+    * Client
+    * Server
+
+* Ausblick
+  * Pi mit VNC (+VPN ;)
+    * Red Pitaya ;)
+  * Alerting
+  * QRZ/LoTW
+
+* Bei Interesse Mail an mich
+  * Useraccount und Link zum Howto kommt