⭐ Dashboard

I haven't quite figured out the difference as well yet. Likely will merge in a single tool.

Although in theory you can plug in any Pandas dataframes in it (assuming they fit the schema).

• low-code/no-code 'platforms' I've seen aren't enough to deal with noisy data or novel usecases.
  
  • filtering outliers, correcting data, etc., is mipossible in a no-code environment
    
  • something like seasonality detection would be impossible without a real programming language
    
• interoperability: if you want some data source that the platform doesn't support, too bad.
  

Many things are very tedious to redo and reinvent from scratch every time

• #datetime handling
  
  • missing dates (e.g. parsing errors)
    
  • some data sources hane timezones, some don't.
    Some are a mixture, which is even worse. E.g. depending on the language you can't sort the dates with mixed timezones.
    

• #errors handling
  Quantified self data is often fuzzy/incomplete etc. If manual inputs are involved, parsing errors are inevitable.
  While in most software it's reasonable to crash on the first error, this would be very annoying if your dashboard breaks of one broken data point!
  On the other hand, you don't want to filter out errors either, since you'd never know they were present in the first place.
  

  So it needs to be something in between, a pattern for propagating errors as far as possible, and handling them gracefully during the visualization phase.
  

• detecting holidays
  E.g. for almost all data it makes sense to differentiate/correlate against days off work
  
• geographical/location data
  

• "so if you want to collaborate feel free to email me"
  

ok, looks kinda slick..
open source: https://github.com/onejgordon/flow-dashboard

Public Github commits
Google Fit - track any activity durations by keyword
Evernote - pull excerpts from specified notebooks
Pocket - Sync stored articles & add notes
Goodreads - Sync currently reading shelf
Track any abstract data via REST API

Heedy is an open-source aggregator built for storage and analysis of your personal data.
It provides a powerful plugin system, allowing easy integration with various services, as well as deep extensibility.

Extensible: Even a system with fantastic visualizations and powerful analysis has limited utility.
This is because it can only show what the original authors assumed would be useful. Heedy offers a powerful plugin system - plugin writers can add new integrations, plots, or even modify core functionality with a few lines of python or javascript.
A registry is planned, so that users can install plugins with the click of a button.

• [2020-08-23] interesing, automatic jupyter notebook connection??
  
• [2021-02-06] yeah looks like very much what I want
  
• [2021-04-27] hmm
  seems that fitbit integration is the only one present so far
  however after giving it a go, it restarted a couple of times, and ended up with a couple of zombie processes
  couldn't restart it without completely tearing down the docker container :(
  Overall not sure where it all fits: there are lots of repositories and not many concrete usecases https://github.com/heedy
  

jupyter notebooks

Sleep data [sleep as android, sleepcycle]
Study hours [toggl]
Exercise [runkeeper, strong]
Programming [wakatime]

interesting, but data exports are not very reusable

the dashboard is currently hard-coded in Google Data Studio

workouts, weight, sleep, hr

https://github.com/jeffshek/open

realtime dashboard

• source: https://github.com/sachac/quantified
  

ok, but just a plot

Blog posts (markdown from github)
Workout data (Strava API)
Recent music (Tidal API)
into one graphql API that can be called from my personal site.

ok, but too low code for me

https://i.redd.it/ufpm4s9k0o821.png

• interact with the backend (so it's possible to reload newer data)
  plotly can do it, bokeh can as well
  
  • ideally, autorefresh?
    
• remember the state (e.g. various panes on/off etc)
  
• be hackable
  
  • via js
    
  • autoreload the code for faster iteradion?
    
• be able to derive new data on the fly? not sure how to achieve this in the browser
  
• have a jupyter interface for better interactivity? ideally, 'edit' any tab as a jupyter notebook
  

https://panel.holoviz.org/

A high-level app and dashboarding solution for Python
Panel works with visualizations from Bokeh, Matplotlib, HoloViews, and many other Python plotting libraries

ok, this is super promising:

Panel can also be used with the separate Param project to create interactively configurable objects with or without associated visualizations, in a fully declarative way. With this approach, you declare your configurable object using the pure-Python, zero-dependency param library, annotating your code with parameter ranges, documentation, and dependencies between parameters and your code. Using this information, you can make all of your domain-specific code be optionally configurable in a GUI, with optional visual displays and debugging information if you like, all with just a few lines of declarations. With this approach, you don’t ever have to decide whether your code will eventually be used in a notebook, in a GUI app, or completely behind the scenes in batch processing, servers, or reports – the same code can support all of these cases equally well, once you declare the associated parameters and constraints.

A Jupyter extensions that turns notebooks into web applications.

there are couple of example apps here
https://www.sicara.ai/blog/2018-01-30-bokeh-dash-best-dashboard-framework-python

bokeh

• seems snappier?
  

plotly:

• possibly better html elements support?
  

pretty sure pandas frames can be easily serialised?

39. Time Series Decomposition Plot

ok, this is pretty intersting
re: autocorrelation – what was up with the 'blue region', significance thing??
also good: Seasonal Plot
would be initersting to quickly toggle it in dashboard?

A modern, enterprise-ready business intelligence web application.

whoa looks interesting…

• [2021-04-26] trying sqlite connection
  

Ugh. it didn't really work – seems to be in the process of deprecation
https://github.com/apache/superset/issues/9748
and still, getting some weird error regardless whether I mount database as read only or not.. and no other logs

2021-04-26 14:19:42,813:WARNING:superset.views.base:[SupersetError(message='(sqlite3.OperationalError) unable to open database file\n(Background on this error at: http://sqlalche.me/e/13/e3q8)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'SQLite', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]

• [2021-04-26] right, managed to run against postgre..
  

a bit finicky overall…
I guess nice that it's possible to separate queries, charts and dashboards…
so one can reuse them in different contexts/for different visualizations

• [2021-04-26]
  for fuck's sake, struggling with creating a 'virtual' view with calculated columns…
  getting an error like
  

  Error: column "_cachew_union_repr_measurement_temp" does not exist
  

• [2021-04-26] seems possible to share queries?
  

https://en.m.wikipedia.org/wiki/Respiratory_rate

For humans, the typical respiratory rate for a healthy adult at rest is 12–18 breaths per minute.[

https://i.reddituploads.com/10f961abe2744c90844287efdd75ba47?fit=max&h=1536&w=1536&s=f019986ae2343e243ed97811b9f500fe
Huh that's a nice way to save on space

Zeo sleep self-experiments
in context

http://vsevolodustinov.ru/blog/all/opyt-obrascheniya-k-somnologu/

. Никакого алкоголя
. Избегать физнагрузку после 17:00. Нагрузка до 17:00 наоборот улучшает сон

try recommendataions from that post and see how they apply to me?

import my.emfit
by_night = my.emfit.by_night()
data = [{'date': x.date, 'hr': x.measured_hr_avg} for x in by_night.values()]
by_dt = pd.DataFrame(data).set_index('date')
by_dt = by_dt.set_index(pd.to_datetime(by_dt.index))
onday = lambda d: (d == by_dt.index.dayofweek)
we = by_dt[onday(0) | onday(1) | onday(6)]
wd = by_dt[onday(3) | onday(4) | onday(5)]
ax = wd.rolling('30D').mean().plot()
we.rolling('30D').mean().plot(ax=ax, color='red')
plt.show()

wonder if it's related to my quality of sleep?

nice, pretty similar to my findings?

exercise will probably have a lot of impact… but then hr is elevated because of the pizza

maybe the data is sifted through the elliptical/running/spinning modules and what's left is just taken as is

e.g. for blood dashboard

e.g. endomondo

merge google and foursquare, display google labels on top of osm mapS
rationale: various tools for working with maps in one place

basically could merge with 4sq? and custom addresses

(although again in theory there might be some seasonality)

in the simplest approach, try to only use cardio and see how it correlates (binary, for instance). although not enough points..

plotly/dash@v1.4.0
Dash v1.4.0

Import time and initialization time should be much improved on Python 3.7 with PR 2368.

https://github.com/plotly/dash/issues/46

from dash import Dash

app = Dash()

app.css.config.serve_locally = True
app.scripts.config.serve_locally = True

Re: [plotly/plotly.py] Importing plotly takes a lot of time (740

https://dash.plot.ly/dash-core-components/tabs

Note that this method has a drawback: it requires that you compute the children property for each individual tab upfront and send all of the tab's content over the network at once. The callback method allows you to compute the tab's content on the fly (that is, when the tab is clicked).

https://github.com/ucg8j/awesome-dash

Only a subset of Bokeh’s objects are capable of rendering in WebGL. Currently supported are the circle and line glyphs, and many markers: asterisk, circle, square, diamond, triangle, inverted_triangle, cross, circle_cross, square_cross, diamond_cross, x, square_x, and circle_x. You can safely combine multiple glyphs in a plot, even if some are rendered in WebGL, and some are not.

some issues: unable to hide zeros —> LOTS of 0 points
unable to attach labels?
influx to start terminal client
lastfm/fill_influxdb.py

http://localhost:3000
influxdb on default port… everything is admin/admin