ixyd
/
wsjtx-dashboards
Archived
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
Browse Source

fix doc

master
Sebastian Denz 4 years ago
parent
7f57611065
commit
5dea1866a7
2 changed files with 49 additions and 43 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 60
      Readme.md
  2. 32
      todo.md

60
Readme.md
View File

@ -3,10 +3,10 @@
![alt text](screenshot.png "Logo Title Text 1")
* a set of tools to export your personal WSJT-X
* live reception data into prometheus or mysql
* historical reception data (since 2019) into mysql
* live reception data into prometheus or mysql
* historical reception data (since 2019) into mysql
* a tool to poll pskreporter and export your personal
* live transmission data to prometheus or mysql
* live transmission data to prometheus or mysql
every time your wsjtx is/was running you are/were generating nice data points, which can be queried using grafana.
this can be very interesting to evaluate your **personal** hf conditions from the current and the past.
@ -35,17 +35,17 @@ have fun!
## tooling overview
* **pskreporter-exporter**
* polls pskreporter.info for your callsign
* supports prometheus and mysql
* polls pskreporter.info for your callsign
* supports prometheus and mysql
* **wsjtx-exporter**
* follows live traffice in ALL.txt
* supports prometheus and mysql
* follows live traffice in ALL.txt
* supports prometheus and mysql
* **alltxt2csv**
* reads whole ALL.txt since 2019 and creates csv-file which can be imported into mysql
* reads whole ALL.txt since 2019 and creates csv-file which can be imported into mysql
* **misc**
* sample dashboards
* shell script to hop between bands via rigctl to allow fractional monitoring of multiple bands
* ready to launch docker-compose setup based on https://github.com/stefanprodan/dockprom
* sample dashboards
* shell script to hop between bands via rigctl to allow fractional monitoring of multiple bands
* ready to launch docker-compose setup based on https://github.com/stefanprodan/dockprom
## things you should be aware of..
@ -53,21 +53,27 @@ have fun!
show pro/con overview:
**prometheus:**
* + you get nicer graphs with counters and bars if you ask me
* + you can use alertmanager if you want advanced alerting
* + optimized for timeseries data
* - not designed for keeping long timeseries (weeks or some months are fine)
* - not possible to import historical data
* - not as flexible as mysql regarding complex queries
**mysql**:
* + you can import your ALL.txt since 2019
* + you can store a lot of data over a long timeframe
* + you can propably build more complex queries in sql
* - in comparison to prometheus not optimized for timeseries, but still good enough
* - gauge based graphs are not as nice as counter/bars if you ask me
* - you can use *only* graphanas internal alerting
### prometheus
pro
* you get nicer graphs with counters and bars if you ask me
* you can use alertmanager if you want advanced alerting
* optimized for timeseries data
con
* not designed for keeping long timeseries (weeks or some months are fine)
* not possible to import historical data
* not as flexible as mysql regarding complex queries
### mysql
pro:
* you can import your ALL.txt since 2019
* you can store a lot of data over a long timeframe
* you can propably build more complex queries in sql
con:
* in comparison to prometheus not optimized for timeseries, but still good enough
* gauge based graphs are not as nice as counter/bars if you ask me
* you can use *only* graphanas internal alerting
both allow distributed setups with multiple wsjtx instances submitting their data to a central prometheus or mysql service.
you can as well run both in parallel and use prometheus for a live overview and mysql for historical evaluations.
@ -110,7 +116,7 @@ choose a bigger interval
### how long does it take to import my data into mysql?
* my ALL.TXT (new format start july 2019) contains ~ 13.7 mio lines and has ~ 850M
* converting to csv takes ~ 14min on i7-4750HQ (2015) and the result has ~ 1.2G
* converting to csv takes ~ 12min on i7-4750HQ (2015) and the result has ~ 1.2G
* currently this is done using another module which uses regular expressions which is not optimial for this use case
* importing the csv to mysql takes ~ 3.5min
* querying the whole time (~ 1.5 years) in grafana takes some seconds

32
todo.md
View File

@ -1,32 +1,32 @@
* wsjtx-exporter
* cqzone und ituzone nicht in db
* -back parameter implementieren
* remove calls metric?
* https://prometheus.io/docs/practices/naming/#labels
* cant reach database ist bei mir aufgetreten
* adressieren ;)
* systemd user unit
* windows aequivalent??
* trace: sucessfully parsed.. loggt: fields.time
* bei kaputter Zeile:
* cqzone und ituzone nicht in db
* -back parameter implementieren
* remove calls metric?
* https://prometheus.io/docs/practices/naming/#labels
* cant reach database ist bei mir aufgetreten
* adressieren ;)
* systemd user unit
* windows aequivalent??
* trace: sucessfully parsed.. loggt: fields.time
* bei kaputter Zeile:
Nov 26 20:10:47 sebo-OptiPlex-980 wsjtx_exporter[869]: goroutine 12 [running]:
Nov 26 20:10:47 sebo-OptiPlex-980 wsjtx_exporter[869]: github.com/denzs/wsjtx_dashboards/shared/wsjtx.ScanLine(0xc0000a4004, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
* alltxt2csv
* direkt importfeature wieder aufnehmen..?
* direkt importfeature wieder aufnehmen..?
* database
* propably more indices
* propably more indices
* create table aus binaries nehmen?
* kontrolle sollte beim db admin liegen
* kontrolle sollte beim db admin liegen
* doc
** german docs..
** server und/oder skript/readme
* german docs..
* server und/oder skript/readme
* fix dashboards
* umgang mit refresh der variablen??
* umgang mit refresh der variablen??
* provide dashboards to grafana