ixyd
/
wsjtx-dashboards
Archived
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
Browse Source

fix doc

master
Sebastian Denz 4 years ago
parent
7f57611065
commit
5dea1866a7
2 changed files with 49 additions and 43 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 60
      Readme.md
  2. 32
      todo.md

60
Readme.md
View File

@ -3,10 +3,10 @@
![alt text](screenshot.png "Logo Title Text 1") ![alt text](screenshot.png "Logo Title Text 1")
* a set of tools to export your personal WSJT-X * a set of tools to export your personal WSJT-X
* live reception data into prometheus or mysql * live reception data into prometheus or mysql
* historical reception data (since 2019) into mysql * historical reception data (since 2019) into mysql
* a tool to poll pskreporter and export your personal * a tool to poll pskreporter and export your personal
* live transmission data to prometheus or mysql * live transmission data to prometheus or mysql
every time your wsjtx is/was running you are/were generating nice data points, which can be queried using grafana. every time your wsjtx is/was running you are/were generating nice data points, which can be queried using grafana.
this can be very interesting to evaluate your **personal** hf conditions from the current and the past. this can be very interesting to evaluate your **personal** hf conditions from the current and the past.
@ -35,17 +35,17 @@ have fun!
## tooling overview ## tooling overview
* **pskreporter-exporter** * **pskreporter-exporter**
* polls pskreporter.info for your callsign * polls pskreporter.info for your callsign
* supports prometheus and mysql * supports prometheus and mysql
* **wsjtx-exporter** * **wsjtx-exporter**
* follows live traffice in ALL.txt * follows live traffice in ALL.txt
* supports prometheus and mysql * supports prometheus and mysql
* **alltxt2csv** * **alltxt2csv**
* reads whole ALL.txt since 2019 and creates csv-file which can be imported into mysql * reads whole ALL.txt since 2019 and creates csv-file which can be imported into mysql
* **misc** * **misc**
* sample dashboards * sample dashboards
* shell script to hop between bands via rigctl to allow fractional monitoring of multiple bands * shell script to hop between bands via rigctl to allow fractional monitoring of multiple bands
* ready to launch docker-compose setup based on https://github.com/stefanprodan/dockprom * ready to launch docker-compose setup based on https://github.com/stefanprodan/dockprom
## things you should be aware of.. ## things you should be aware of..
@ -53,21 +53,27 @@ have fun!
show pro/con overview: show pro/con overview:
**prometheus:** ### prometheus
* + you get nicer graphs with counters and bars if you ask me
* + you can use alertmanager if you want advanced alerting pro
* + optimized for timeseries data * you get nicer graphs with counters and bars if you ask me
* - not designed for keeping long timeseries (weeks or some months are fine) * you can use alertmanager if you want advanced alerting
* - not possible to import historical data * optimized for timeseries data
* - not as flexible as mysql regarding complex queries con
* not designed for keeping long timeseries (weeks or some months are fine)
**mysql**: * not possible to import historical data
* + you can import your ALL.txt since 2019 * not as flexible as mysql regarding complex queries
* + you can store a lot of data over a long timeframe
* + you can propably build more complex queries in sql ### mysql
* - in comparison to prometheus not optimized for timeseries, but still good enough
* - gauge based graphs are not as nice as counter/bars if you ask me pro:
* - you can use *only* graphanas internal alerting * you can import your ALL.txt since 2019
* you can store a lot of data over a long timeframe
* you can propably build more complex queries in sql
con:
* in comparison to prometheus not optimized for timeseries, but still good enough
* gauge based graphs are not as nice as counter/bars if you ask me
* you can use *only* graphanas internal alerting
both allow distributed setups with multiple wsjtx instances submitting their data to a central prometheus or mysql service. both allow distributed setups with multiple wsjtx instances submitting their data to a central prometheus or mysql service.
you can as well run both in parallel and use prometheus for a live overview and mysql for historical evaluations. you can as well run both in parallel and use prometheus for a live overview and mysql for historical evaluations.
@ -110,7 +116,7 @@ choose a bigger interval
### how long does it take to import my data into mysql? ### how long does it take to import my data into mysql?
* my ALL.TXT (new format start july 2019) contains ~ 13.7 mio lines and has ~ 850M * my ALL.TXT (new format start july 2019) contains ~ 13.7 mio lines and has ~ 850M
* converting to csv takes ~ 14min on i7-4750HQ (2015) and the result has ~ 1.2G * converting to csv takes ~ 12min on i7-4750HQ (2015) and the result has ~ 1.2G
* currently this is done using another module which uses regular expressions which is not optimial for this use case * currently this is done using another module which uses regular expressions which is not optimial for this use case
* importing the csv to mysql takes ~ 3.5min * importing the csv to mysql takes ~ 3.5min
* querying the whole time (~ 1.5 years) in grafana takes some seconds * querying the whole time (~ 1.5 years) in grafana takes some seconds

32
todo.md
View File

@ -1,32 +1,32 @@
* wsjtx-exporter * wsjtx-exporter
* cqzone und ituzone nicht in db * cqzone und ituzone nicht in db
* -back parameter implementieren * -back parameter implementieren
* remove calls metric? * remove calls metric?
* https://prometheus.io/docs/practices/naming/#labels * https://prometheus.io/docs/practices/naming/#labels
* cant reach database ist bei mir aufgetreten * cant reach database ist bei mir aufgetreten
* adressieren ;) * adressieren ;)
* systemd user unit * systemd user unit
* windows aequivalent?? * windows aequivalent??
* trace: sucessfully parsed.. loggt: fields.time * trace: sucessfully parsed.. loggt: fields.time
* bei kaputter Zeile: * bei kaputter Zeile:
Nov 26 20:10:47 sebo-OptiPlex-980 wsjtx_exporter[869]: goroutine 12 [running]: Nov 26 20:10:47 sebo-OptiPlex-980 wsjtx_exporter[869]: goroutine 12 [running]:
Nov 26 20:10:47 sebo-OptiPlex-980 wsjtx_exporter[869]: github.com/denzs/wsjtx_dashboards/shared/wsjtx.ScanLine(0xc0000a4004, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) Nov 26 20:10:47 sebo-OptiPlex-980 wsjtx_exporter[869]: github.com/denzs/wsjtx_dashboards/shared/wsjtx.ScanLine(0xc0000a4004, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
* alltxt2csv * alltxt2csv
* direkt importfeature wieder aufnehmen..? * direkt importfeature wieder aufnehmen..?
* database * database
* propably more indices * propably more indices
* create table aus binaries nehmen? * create table aus binaries nehmen?
* kontrolle sollte beim db admin liegen * kontrolle sollte beim db admin liegen
* doc * doc
** german docs.. * german docs..
** server und/oder skript/readme * server und/oder skript/readme
* fix dashboards * fix dashboards
* umgang mit refresh der variablen?? * umgang mit refresh der variablen??
* provide dashboards to grafana * provide dashboards to grafana