dashboard

⭐ Dashboard

Dashboard (not a very unique name, I know) is a set of tools to visualize personal data on a, well, some sort of dashboard.

I'm gradually publishing it here and got some old screenshots here.

Ideally I'd like to reuse existing libraries and other people's projects to the maximum extent possible, so we could join forces.

Table of Contents

related dashboard

used to visualize #qs data dashboardqs

very similar to #timeline and #memex dashboardtimelinememex

I haven't quite figured out the difference as well yet. Likely will merge in a single tool.

#hpi provides data for my dashboard dashboardhpi

Although in theory you can plug in any Pandas dataframes in it (assuming they fit the schema).

[A] * motivation dashboard

  • same motivation as for quantified self
  • a 'research' goal: figuring out the patterns and common routines for processing and preparing this data for visualizing.

[A] why not use something existing? dashboard

  • low-code/no-code 'platforms' I've seen aren't enough to deal with noisy data or novel usecases.
    • filtering outliers, correcting data, etc., is mipossible in a no-code environment
    • something like seasonality detection would be impossible without a real programming language
  • interoperability: if you want some data source that the platform doesn't support, too bad.

[B] figuring out core patterns for manipulating and representing this data dashboard

Many things are very tedious to redo and reinvent from scratch every time

  • #datetime handling
    • missing dates (e.g. parsing errors)
    • some data sources hane timezones, some don't.
      Some are a mixture, which is even worse. E.g. depending on the language you can't sort the dates with mixed timezones.
  • #errors handling
    Quantified self data is often fuzzy/incomplete etc. If manual inputs are involved, parsing errors are inevitable.
    While in most software it's reasonable to crash on the first error, this would be very annoying if your dashboard breaks of one broken data point!
    On the other hand, you don't want to filter out errors either, since you'd never know they were present in the first place.

    So it needs to be something in between, a pattern for propagating errors as far as possible, and handling them gracefully during the visualization phase.

  • detecting holidays
    E.g. for almost all data it makes sense to differentiate/correlate against days off work
  • geographical/location data

[B] automatically detecting interesting correlations between data dashboard

TODO [B] [2019-08-15] wonder if massive dataframe with literally everything could help dashboard

TODO maybe just go over all series? and then generate all pairwise correlations and sort by correlation coefficient + R2? some sort of pareto surface? dashboard

[B] what I want to figure out: dashboard

TODO [B] are 'sleep phases' a thing (or at least the way they are measured in conventional tackers)? how do they impact my waking up, alertness etc? dashboardsleep

CREATED: [2021-02-04]

TODO [C] [2020-10-27] calculating custom exercise stats dashboard

e.g. I have a few traffic lights on my way that might affect my speed/cadence data, and I might want to filter it out

[C] custom/personalized data processing dashboard

TODO [C] splitting exercise into cardio and non-cardio dashboard

CREATED: [2020-10-18]
  • with cardio (at least, endurance styl) there is a somewhat convincing metric – number of heartbeats, which is basically the same as calories burnt?
  • with strength training this is something very hard to measure and not sure if makes sense at all?

TODO [B] demo of errors: typical mistake is confuising wlog and weight tags (weight gets unrecognized as exercise) dashboard

CREATED: [2021-02-18]

[A] * inspiration dashboard

STRT [B] [2018-06-06] andreilyskov: qs dashboard example dashboardqs

  • "so if you want to collaborate feel free to email me"

[2019-04-06] interesting dashboards: spotify, writing and reading. otherwise not that much stuff dashboardqs

[2020-08-22] ok seems that they use similar architecture to mine https://forum.quantifiedself.com/t/personal-dashboards-for-self-tracking-data/8202/4 dashboardqs

STRT [B] [2020-08-22] Flow Dashboard dashboard

ok, looks kinda slick..
open source: https://github.com/onejgordon/flow-dashboard

Public Github commits
Google Fit - track any activity durations by keyword
Evernote - pull excerpts from specified notebooks
Pocket - Sync stored articles & add notes
Goodreads - Sync currently reading shelf
Track any abstract data via REST API

TODO [2021-02-06] ok, rest api thing is cool, should try connecting it to #hpi dashboardhpi

[C] [2019-11-24] heedy/heedy: An Open-Source Platform for Quantified Self & IoT dashboard

Heedy is an open-source aggregator built for storage and analysis of your personal data.
It provides a powerful plugin system, allowing easy integration with various services, as well as deep extensibility.
Extensible: Even a system with fantastic visualizations and powerful analysis has limited utility.
This is because it can only show what the original authors assumed would be useful. Heedy offers a powerful plugin system - plugin writers can add new integrations, plots, or even modify core functionality with a few lines of python or javascript.
A registry is planned, so that users can install plugins with the click of a button.
  • [2020-08-23] interesing, automatic jupyter notebook connection??
  • [2021-02-06] yeah looks like very much what I want
  • [2021-04-27] hmm
    seems that fitbit integration is the only one present so far
    however after giving it a go, it restarted a couple of times, and ended up with a couple of zombie processes
    couldn't restart it without completely tearing down the docker container :(
    Overall not sure where it all fits: there are lots of repositories and not many concrete usecases https://github.com/heedy

STRT [C] [2019-05-21] quelf/report.Rmd at master · JakobGM/quelf dashboard

jupyter notebooks

Sleep data [sleep as android, sleepcycle]
Study hours [toggl]
Exercise [runkeeper, strong]
Programming [wakatime]

STRT [C] [2019-09-22] markwk/qsledger: Quantified Self Personal Data Aggregator and Data Analysis dashboardqsdataliberation

interesting, but data exports are not very reusable

[2019-09-28] I guess once I export everything could send a link to that guy? dashboardqsdataliberation

STRT [C] [2020-09-03] https://github.com/KrauseFx/FxLifeSheet dashboard

the dashboard is currently hard-coded in Google Data Studio

TODO [C] [2020-08-22] some good dashboards https://forum.quantifiedself.com/t/personal-dashboards-for-self-tracking-data/8202/15 dashboard

workouts, weight, sleep, hr

TODO [C] Juno's Personal Data Exploratory https://exploratory.openhumans.org/notebooks dashboardpkmqs

CREATED: [2019-12-19]

STRT [C] [2020-07-02] jeffshek/betterself: Your body's dashboard. dashboard

[2020-08-24] demo https://app.betterself.io dashboard

  • table with notes is kinda nice?
    eh. would be nice to have something automatic for dataframes

STRT [C] [2019-01-13] Sacha Chua: quantified awesome dashboardqs

realtime dashboard

[2019-04-06] ok, looks pretty clean, but might require some tinkering dashboardqs

[2021-02-06] also pretty much only time tracking? dashboardqs

STRT [D] [2020-04-04] ammanvedi/quantified-self-server: Aggregate data about myself (workouts, blog posts, music listening history) into a graphql API dashboard

Blog posts (markdown from github)
Workout data (Strava API)
Recent music (Tidal API)
into one graphql API that can be called from my personal site.

CANCEL [D] [2019-04-19] markwk: How to Create a Time Tracking Dashboard using RescueTime, IFTTT and Google Sheets dashboardqs

ok, but too low code for me

CANCEL [D] [2019-01-08] Personal Dashboard /r/QuantifiedSelf dashboard

https://i.redd.it/ufpm4s9k0o821.png

[2019-04-11] it is not very elaborate though… dashboard

[B] minor ideas dashboard

TODO [B] keeping query in the address string is pretty clever… can bookmark and easily restart it! dashboarddatasette

CREATED: [2020-11-25]

TODO [C] use tabs within tabs? kinda like in garmin https://connect.garmin.com/modern/activities dashboard

CREATED: [2020-10-15]

TODO [C] [2020-10-27] Garmin Connect dashboard

pretty nice clean visualizations, I need to borrow these I guess

[B] * implementation dashboard

How to actuallly implement it, which software/tools/libraries to use.

TODO [A] (meta?)goals dashboard

CREATED: [2020-08-22]
  • interact with the backend (so it's possible to reload newer data)
    plotly can do it, bokeh can as well
    • ideally, autorefresh?
  • remember the state (e.g. various panes on/off etc)
  • be hackable
    • via js
    • autoreload the code for faster iteradion?
  • be able to derive new data on the fly? not sure how to achieve this in the browser
  • have a jupyter interface for better interactivity? ideally, 'edit' any tab as a jupyter notebook

TODO keep it modular, I guess dashboard

CREATED: [2020-08-22]

have a core subpackages and various tabs… not sure how to refactor them out later

[2020-09-01] similarly to HPI, it's more of a demonstration how it can be done. Ideally people would be able to reuse core and build their own dashboards dashboard

TODO [A] it's very important to benefit from the existing data science infrastructure as much as possible. jupyter, pandas, etc. dashboard

CREATED: [2020-10-01]

STRT [A] [2020-12-10] panel · PyPI dashboard

https://panel.holoviz.org/

A high-level app and dashboarding solution for Python
Panel works with visualizations from Bokeh, Matplotlib, HoloViews, and many other Python plotting libraries

ok, this is super promising:

Panel can also be used with the separate Param project to create interactively configurable objects with or without associated visualizations, in a fully declarative way. With this approach, you declare your configurable object using the pure-Python, zero-dependency param library, annotating your code with parameter ranges, documentation, and dependencies between parameters and your code. Using this information, you can make all of your domain-specific code be optionally configurable in a GUI, with optional visual displays and debugging information if you like, all with just a few lines of declarations. With this approach, you don’t ever have to decide whether your code will eventually be used in a notebook, in a GUI app, or completely behind the scenes in batch processing, servers, or reports – the same code can support all of these cases equally well, once you declare the associated parameters and constraints.

STRT [A] [2020-12-07] oschuett/appmode: A Jupyter extensions that turns notebooks into web applications. dashboard

A Jupyter extensions that turns notebooks into web applications.

[2021-02-06] ok, nice, has 'edit app' button which opens jupyter to edit?? dashboard

[B] [2020-08-23] ok, #jupyter is the ultimate ad-hoc solution. it preserves state, can be exposed as a frontend and allows for python code dashboard

STRT [B] bokeh vs plotly dash dashboard

CREATED: [2020-08-22]

there are couple of example apps here
https://www.sicara.ai/blog/2018-01-30-bokeh-dash-best-dashboard-framework-python

bokeh

  • seems snappier?

plotly:

  • possibly better html elements support?

[2020-09-03] compromise: keep it as framework independent and decoupled as possible. Support both frameworks! dashboard

STRT [B] basically have a core library that's imported in all notebooks. move stuff to core as long as possible, but allow quick button to mess with the data in notebook dashboardtoblog

CREATED: [2020-08-24]

TODO [B] try jupyterlab dashboard

CREATED: [2020-11-06]

TODO [B] I think dashboard needs to preserve state somehow, it's annoying to edit code every time.. wonder if dash supports it? dashboard

CREATED: [2020-07-28]

TODO [C] use multiprocessing… dashboard

CREATED: [2019-07-24]

pretty sure pandas frames can be easily serialised?

TODO [C] [2020-09-10] Top 50 matplotlib Visualizations - The Master Plots (w/ Full Python Code) | ML+ dashboard

39. Time Series Decomposition Plot

ok, this is pretty intersting
re: autocorrelation – what was up with the 'blue region', significance thing??
also good: Seasonal Plot
would be initersting to quickly toggle it in dashboard?

STRT [C] grafana could simply be one of the interfaces dashboard

CREATED: [2020-08-25]

STRT [B] [2021-02-17] apache/superset: Apache Superset is a Data Visualization and Data Exploration Platform dashboardhpi

A modern, enterprise-ready business intelligence web application.

whoa looks interesting…

  • [2021-04-26] trying sqlite connection

Ugh. it didn't really work – seems to be in the process of deprecation
https://github.com/apache/superset/issues/9748
and still, getting some weird error regardless whether I mount database as read only or not.. and no other logs

2021-04-26 14:19:42,813:WARNING:superset.views.base:[SupersetError(message='(sqlite3.OperationalError) unable to open database file\n(Background on this error at: http://sqlalche.me/e/13/e3q8)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'SQLite', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
  • [2021-04-26] right, managed to run against postgre..

a bit finicky overall…
I guess nice that it's possible to separate queries, charts and dashboards…
so one can reuse them in different contexts/for different visualizations

  • [2021-04-26]
    for fuck's sake, struggling with creating a 'virtual' view with calculated columns…
    getting an error like

    Error: column "_cachew_union_repr_measurement_temp" does not exist
    
  • [2021-04-26] seems possible to share queries?

[B] * sleep dashboard/experiments dashboardsleep

Sleep is one of the biggest things I wanna figure out, and I have quite a few ideas of what to test.

TODO [B] Maybe run pca for my sleep data? dashboardsleepqsml

CREATED: [2019-10-20]

TODO [B] figure out main questions I wanna figure out dashboardsleepqs

CREATED: [2019-08-03]

e.g. 'what is the best time for me to go asleep'? dashboardqssleep

'what is the best sleep duration?' dashboardqssleep

STRT [C] [2019-08-21] Respiratory rate - Wikipedia dashboardsleep

https://en.m.wikipedia.org/wiki/Respiratory_rate

For humans, the typical respiratory rate for a healthy adult at rest is 12–18 breaths per minute.[

[2020-08-24] add to dataframe dashboardsleep

TODO [B] correlate with sliding exponential mean instead? dashboardsleep

CREATED: [2019-04-17]

STRT [B] add date to tooltip dashboardsleep

CREATED: [2019-07-22]

STRT [B] def need to highlight holidays on the background (also annotate if it was indeed a day off), e.g. where have I gone dashboardsleep

CREATED: [2020-08-25]

TODO [C] wonder if coverage is correlated with sleep movement… dashboardsleep

CREATED: [2019-04-07]

TODO [C] find correlation between bedtime and length of sleep? dashboardsleep

CREATED: [2019-04-07]

TODO [C] Sleep/exercise correlation: could try different deltas/correlation coeff plot dashboardsleep

CREATED: [2019-04-08]

TODO [C] old sleep logs from taplog? dashboardsleep

CREATED: [2019-04-09]

TODO [C] check waking up in REM? dashboardsleep

CREATED: [2019-04-09]

TODO [C] interesting correlation: sleep vs bed time. wonder if that means REM or something? dashboardsleep

CREATED: [2019-07-22]

TODO [C] hmm, sleepy correlates negatively with ex. volume? dashboardsleep

CREATED: [2019-07-23]

TODO [C] hmm. temperature is interesting – looks like it's negatively correlating with HR?? dashboardsleepqs

CREATED: [2019-08-04]

TODO [C] sleep plot: highlight holidays/weekends as background color? dashboardsleep

CREATED: [2019-12-08]

TODO [C] movements/bed exit in emfit? dashboardsleep

CREATED: [2019-08-21]

[2020-08-25] could plot on the sleep bars plot? dashboardsleep

TODO [C] integrate old melatonin experiment, maybe post about it dashboardsleep

CREATED: [2019-08-20]

TODO [C] use ANS balance from emfit? dashboardsleep

CREATED: [2019-04-09]

[C] [2018-06-20] Emfit qs estimated my sleep high, even though i felt a bit like shit. I wonder if it just correlates with sleep length.. dashboardsleep

TODO [C] temperature during sleep dashboardsleepqs

CREATED: [2019-08-04]

TODO [B] different style for sleep length bars? currently red and blue are sort of annoyingly make it hard to spot the pattern? dashboardsleepemfit

CREATED: [2020-08-21]

TODO [C] Average temperature dashboardsleep

CREATED: [2019-04-09]

TODO [C] cross correlate with rescuetime activity dashboardsleep

CREATED: [2020-08-22]

[2020-08-25] could even plot activity bars dashboardsleep

TODO [C] figure out alarm wakeup vs 'natural' wakeup, could tell me something about sleep intervals dashboardsleepqs

CREATED: [2019-08-26]

STRT [D] Emfit recovery – wonder if it correlated with exercise days dashboardsleepemfit

CREATED: [2018-06-28]

TODO [B] My daughters sleeping patterns for the first 4 months of her life. One continuous spiral starting on the inside when she was born, each revolution representing a single day. Midnight at the top (24 hour clock). [OC] dashboardsleepinspirationviz

TODO [C] Melatonin analysis dashboardsleep

CREATED: [2019-12-05]

Zeo sleep self-experiments
in context

TODO [B] wonder what does it mean when morning hrv is higher than evening?? dashboardsleepqshrv

CREATED: [2020-09-02]

TODO [C] I wonder what happened… at some point (around august 2019), coverage has consistently gone up to 100 dashboardsleepqsemfit

CREATED: [2020-08-22]

TODO [B] Track pizza dependency? dashboardsleepqs

CREATED: [2019-09-23]

TODO [B] Shit wonder if I'm oversleeping because it's too cold? dashboardsleepqs

CREATED: [2019-10-15]

TODO [B] pretty strong negative correlation of temp vs avg hr. wonder if that's searsonal or not? post about it soon? dashboardsleepqs

CREATED: [2019-11-17]

TODO [B] look at HRV peaks and try to see what's happening? dashboardsleepqstimeline

CREATED: [2019-11-17]

[B] [2019-11-14] Опыт обращения к сомнологу dashboardsleepqs

http://vsevolodustinov.ru/blog/all/opyt-obrascheniya-k-somnologu/

. Никакого алкоголя
. Избегать физнагрузку после 17:00. Нагрузка до 17:00 наоборот улучшает сон

try recommendataions from that post and see how they apply to me?

TODO [B] Compare hr/hrv before and after holidays dashboardsleepqshpi

CREATED: [2019-12-01]

TODO [B] Compute differences between rem cycles and plot on histogram? dashboardsleepqs

CREATED: [2020-01-07]

TODO [B] wow, emfit recovery was really shit last night. why???… seriosly no reason for it dashboardsleepqs

CREATED: [2020-03-25]

[B] wip on figuring out if weekdays impact sleep dashboardsleepqs

CREATED: [2020-05-03]
import my.emfit
by_night = my.emfit.by_night()
data = [{'date': x.date, 'hr': x.measured_hr_avg} for x in by_night.values()]
by_dt = pd.DataFrame(data).set_index('date')
by_dt = by_dt.set_index(pd.to_datetime(by_dt.index))
onday = lambda d: (d == by_dt.index.dayofweek)
we = by_dt[onday(0) | onday(1) | onday(6)]
wd = by_dt[onday(3) | onday(4) | onday(5)]
ax = wd.rolling('30D').mean().plot()
we.rolling('30D').mean().plot(ax=ax, color='red')
plt.show()

TODO [B] plot my emfit sleep, to be fair even two years of plots is quite cool dashboardsleeppublishqs

CREATED: [2020-05-03]

TODO [B] hmm, my hrv really has fallen down? also sleep hr gone up dashboardsleepqshealth

CREATED: [2020-06-22]

wonder if it's related to my quality of sleep?

TODO [D] track dependency on meditaion? dashboardsleep

CREATED: [2019-04-09]

TODO [C] display times in bed definitely… dashboardsleep

CREATED: [2020-09-10]

TODO would be nice to plot against sunrise/sunset? dashboardsleep

CREATED: [2020-09-19]

TODO [2020-09-30] fucking hell! after naples HRV jumped and HR dropped very significantly dashboardsleepself

TODO [B] correlate dreams with rem/deep etc? dashboardsleep

CREATED: [2020-10-10]

TODO [B] wtf… without seasons it seems to correlate less?? dashboardsleep

CREATED: [2020-10-23]

TODO [C] negative correlation of coverage with temperature is pretty weird dashboardsleepemfit

CREATED: [2020-10-11]

TODO [C] Compare fitbit dashboardsleephrqs

CREATED: [2017-12-21]

STRT [B] For sleep, probably a good idea to log room temperature dashboardsleepqs

CREATED: [2017-11-13]

TODO [B] hmm, maybe I need to remove seasonality from the sleep first and then search other correlations?? not sure.. dashboardsleep

CREATED: [2020-10-22]

TODO [B] ugh. today is gonna be confusing dashboardsleepqs

CREATED: [2020-10-09]

exercise will probably have a lot of impact… but then hr is elevated because of the pizza

[2020-10-18] maybe really need to try two-parameter regression? dashboardqssleep

[2020-10-18] take into the account past pizza orders? this is gonna have systematic impact on my sundays? dashboardqssleep

[B] * exercise dashboard/experiments dashboard

TODO [B] running dashboard – custom dataframe + add distance + merge with manual notes in a similar way dashboard

CREATED: [2020-09-14]

TODO [B] thinking how to contribute data for the energy plot dashboardexerciseqs

CREATED: [2020-09-15]

maybe the data is sifted through the elliptical/running/spinning modules and what's left is just taken as is

TODO [B] have some stuff in endomondo comments… need to extract it and use. maybe actually just reuse endoexport, and put in a df dashboardqs

CREATED: [2020-09-14]

TODO [C] Show distance on speed/hr plot dashboardqs

CREATED: [2019-09-25]

STRT [B] def need to measure decay to 60 to get more data… dashboardqshrtoblog

CREATED: [2019-08-16]

STRT [B] process google location and guess walks from it dashboardexerciseqs

CREATED: [2020-08-01]

TODO [B] Detect walks and runs from google data and plot them. Mark velocity with colours. Could also do 'walking intervals' (e.g. if was in a shop inbetween) dashboardexercise

CREATED: [2020-08-23]

TODO [C] calculate some sort of 'integral' heart rate and also score for interval trainings? dashboardexercise

CREATED: [2019-04-09]

[2020-09-06] eh? maybe I wanted to break down my interval trainings per activity bursts? dashboardexercise

STRT [C] integrate elliptical workouts? Could plot workout specific things and make them easy to compare dashboard

CREATED: [2019-07-23]

STRT [C] filter out specific exercises and plot separately to see the progress dashboardexercise

CREATED: [2020-10-13]

TODO [C] go through the most common exercise (past) sessions and check them dashboardexercise

CREATED: [2020-10-15]

[2021-02-06] I guess it's a more general pattern for reviewing data too dashboardexercise

TODO [B] exercise provider: trust more notes which got HR data dashboardqs

CREATED: [2018-05-11]

STRT [C] Look at hr data, its almost 180 all the time. Is it because i was doing cardio less? dashboardqshealthexercise

CREATED: [2019-01-05]

[C] * patterns dashboard

as in, patterns of working with data, best practices for visualizing things etc

TODO [A] use dynamic sliders for different regression windows dashboard

CREATED: [2019-07-30]

STRT [B] Always plot isolated metrics (temp, hr, etc) – good for debugging dashboardqs

CREATED: [2020-06-26]

[2020-09-03] also make sure it's easy to do dashboardqs

TODO [B] restrict datapoints 'by experiment'? same way it's done with date sliders? dashboard

CREATED: [2019-07-24]

STRT [B] need to try different date shifts to see it's not correlated dashboard

CREATED: [2019-04-09]

TODO [B] labels: need to display weekday (also on HR nodes?) dashboard

CREATED: [2019-11-23]

TODO [B] would be nice to zoom individual plots.. dashboard

CREATED: [2020-10-09]

[B] * error hanling dashboarderrors

TODO [C] about importance of error handling when working with personal data dashboarderrorstoblog

CREATED: [2020-09-14]

data is inherently messy and error prone
it's sensible to be defensive and try to relax as much as possible (unless you have time to literally tend to them immediately)
on the other hand, you want to make sure you are aware about the issues in data
example: running dashboard – has a warning about 0 speed
see screenshot at the same time (fix JS table first so duration & starttime have proper rendering)
you can instantly see it on the plot (yellow markers). if you hover, you'll see that the error is about zero speed (and you can also see it in the table)
in the table, you can clearly spot that the problem is the treadmill – indeed, treadmill wasn't connected to the phone in any way, and I'm stationary, so endomondo has no idea how much I moved
however, I did log the distance (along with the training regime) manually
by combining the HR data (recorded by endomondo) and manual data we get the complete representation
(if I add visual highlight on the table, that would be fucking amazing)

  • [2020-09-14] maybe post on QS forum?? dashboarderrorstoblog

TODO [C] error handling – demonstrate on 'multiple sleeps' dashboarderrors

CREATED: [2020-09-12]

TODO [C] demo: running speed/hr plot is a good candidate dashboarderrors

CREATED: [2020-09-16]
  • show both types of errors (zero speed/no HR)
  • need to disable tooltip on avg plot
  • would be nice to highlight the point under the tooltip?

TODO [C] demo: unmark one of cardio/non-cardio and demonstrate warnings dashboarderrors

CREATED: [2020-09-17]

TODO [C] document error handling pattern with 'error' dashboarderrorstoblog

CREATED: [2020-09-07]

TODO [B] for correlations, there is overview + zoomed in individual plots showing all the errors, outliers, etc dashboard

CREATED: [2020-10-18]

TODO [B] Each rolling plot can also handle seasonality, residuals etc? dashboard

CREATED: [2020-10-23]

TODO [B] would be cool to scale arbitrarily vertically and horizontally, the whole plot dashboard

CREATED: [2020-09-10]

TODO [B] indicate how recent datapoints are by color? (in the correlations) dashboard

CREATED: [2019-04-09]

TODO [C] corr plot: include influence/leverage etc dashboard

CREATED: [2020-10-10]

STRT [C] would be nice to display data source? dashboard

CREATED: [2020-08-22]

e.g. for blood dashboard

TODO [B] [2019-07-20] links to event sources (e.g. emfit etc) dashboard

  • [2020-09-20] maybe could be automatic/agnostic? add to cdf check? dashboard

TODO [C] [2019-04-12] src in exercise volume dashboard

TODO [C] add src for jawbone/emfit sleep dashboard

CREATED: [2020-10-21]

TODO [C] link to original event from point? or just 'context' dashboard

CREATED: [2019-08-25]

e.g. endomondo

TODO [C] vertical line tooltip sucks… probably need a single point instead dashboard

CREATED: [2020-10-24]

TODO [C] definitely need to take seasonality (e.g. weekly) into the account, e.g. for multi regression dashboard

CREATED: [2020-10-19]

TODO [C] I guess need seasonality analisys for each metric? maybe have it on a separate pane, e.g. 'debug/insights'? would be nice to do it for all dataframes? dashboard

CREATED: [2020-10-23]

TODO [C] would be nice to include the org file link to the context… then could use mimemacs for it dashboardpromnesia

CREATED: [2020-10-16]

[C] * ideas for more dashboards dashboard

TODO [B] rescuetime summarizes my sleep intervals pretty well? could at least have upper bound on sleep during holidays/when not using emfit? dashboardrescuetimehpi

CREATED: [2020-09-18]

STRT [C] temperature – probably, seasonality? dashboardqs

CREATED: [2019-08-04]

TODO [C] display photos as an optional layer along with the map? dashboard

CREATED: [2020-08-24]

[2020-09-06] just need to integrate dashboard with photomap dashboard

TODO [C] links into food dashboards.. I guess I need anchors. dashboardnutrino

TODO [C] favsmap should be part of dashboard… also could be the first bit in my public dashboard? dashboard

CREATED: [2020-08-02]

merge google and foursquare, display google labels on top of osm mapS
rationale: various tools for working with maps in one place

TODO [C] shit. if I had food stats that would be just amazing dashboard

CREATED: [2019-04-09]

TODO [D] takeout – contains list of places dashboardtakeouthpi

CREATED: [2020-09-01]

basically could merge with 4sq? and custom addresses

TODO [D] money? not sure dashboard

CREATED: [2020-08-22]

TODO [D] quantified mind? already have plots dashboard

CREATED: [2019-08-20]

TODO [D] Add check-ins to favsmap dashboard

CREATED: [2019-11-28]

[C] * ideas for experiments dashboard

TODO [B] correlate number of commits/lines vs sleep? dashboard

CREATED: [2020-10-21]

TODO [B] glucose and ketone measurements vs food? dashboard

CREATED: [2019-04-09]

[2020-09-06] probably not enough data… dashboard

TODO [C] correlate temp sensor with hiking? dashboardtoblogqslifelogging

CREATED: [2019-10-30]

TODO [C] hmm, maybe need to try running/eating pizza on a different day? to make sure the seasonality is not due to running? dashboardexerciseqssleep

CREATED: [2020-10-24]

[2020-10-24] or could check seasonality for months that didn't involve running? dashboardexerciseqssleep

TODO [C] volume vs avg temp is an interesting one to demonstrate the correlation that shouldn't exist dashboard

CREATED: [2020-10-19]

(although again in theory there might be some seasonality)

DONE [C] try to find correlation between sleep and exercise? dashboardqs

CREATED: [2019-02-24]
  • State "DONE" from "TODO" [2019-04-06]

in the simplest approach, try to only use cardio and see how it correlates (binary, for instance). although not enough points..

TODO [C] temperature plots: would be interesting to have avg plot for 'indoors' and 'outdoors' periods? dashboard

CREATED: [2020-08-23]

TODO [D] plot spinning power vs heartbeats? dashboardqs

CREATED: [2019-11-23]

[D] * plotly dash dashboardplotly

For now using #bokeh instead

[C] [2019-10-27] plotly/dash@1.4.0 dashboardplotly

plotly/dash@v1.4.0
Dash v1.4.0

[C] [2020-04-10] Re: [plotly/plotly.py] Importing plotly takes a lot of time (740) dashboardplotly

Import time and initialization time should be much improved on Python 3.7 with PR 2368.

[C] [2019-04-10] Dash offline is enabled? plotly/dash dashboardplotly

https://github.com/plotly/dash/issues/46

from dash import Dash

app = Dash()

app.css.config.serve_locally = True
app.scripts.config.serve_locally = True

TODO [C] wow. showing points on other plots while rectangle selection is really awesome dashboardplotly

CREATED: [2019-07-22]

STRT [C] would be really nice to properly render pairplots in plotly dashboardplotly

CREATED: [2019-04-07]

TODO [C] err. seems to always consume cpu in background dashboardplotly

CREATED: [2019-04-08]

[C] [2020-08-26] Re: plotly/plotly.py Importing plotly takes a lot of time (740) dashboardplotly

Re: [plotly/plotly.py] Importing plotly takes a lot of time (740

[D] [2019-04-07] Dash User Guide and Documentation - Dash by Plotly dashboardplotly

https://dash.plot.ly/dash-core-components/tabs

Note that this method has a drawback: it requires that you compute the children property for each individual tab upfront and send all of the tab's content over the network at once. The callback method allows you to compute the tab's content on the fly (that is, when the tab is clicked).

DONE [B] [2019-05-30] ucg8j/awesome-dash: A curated list of awesome Dash (plotly) resources dashboardplotlyviz

[2019-07-24] https://dash-gallery.plotly.host/Portal/ dashboardplotlyviz

[2019-07-24] eh, dunno otherwise not so interesting? maybe I'm not really lacking anything, I can think of many things I can add that still need to be implemented.. dashboardplotlyviz

----------------------------------------- dashboard

TODO [A] come up with a better name? just in case of pypi release… dashboard

CREATED: [2020-08-22]

STRT [B] how to detect temporal correlation? dashboardstudyqs

CREATED: [2018-02-28]

[2020-09-01] ok, this lag thing in bokeh could work dashboardstudyqs

[B] [2019-04-19] qsledger/examplecorrelationexplorerwithplotly.py at master · markwk/qsledger dashboard

TODO [B] [2019-10-15] Why you should continuously track your energy level and what I've learned from it. /r/Biohackers dashboardqs

[2019-10-18] about tracker matching subjective score dashboardqs

TODO [B] not sure what mavg is doing. e.g. HR plots of specific exercise – if I didn't exercise during the past month mavg for 14 days should just take value of my last exercise. right?? dashboard

CREATED: [2019-08-15]

[2020-09-03] need to test it, just to double check dashboard

TODO [B] [2019-04-11] Visualization — pandas 0.24.2 documentation dashboardviz

TODO [B] [2020-09-16] Building a Data Dashboard - Quantified Self / Apps & Tools - Quantified Self Forum dashboard

TODO [B] [2020-08-23] All | Search powered by Algolia dashboard

Panel, hvPlot, HoloViews, GeoViews, Datashader, Param, Colorcet -- all working together to make Python data visualization easier and more powerful.

TODO [B] [2020-10-11] Accelerating with WebGL — Bokeh 2.2.1 Documentation dashboard

[2020-10-11] Accelerating with WebGL — Bokeh 2.2.1 Documentation dashboard

Only a subset of Bokeh’s objects are capable of rendering in WebGL. Currently supported are the circle and line glyphs, and many markers: asterisk, circle, square, diamond, triangle, inverted_triangle, cross, circle_cross, square_cross, diamond_cross, x, square_x, and circle_x. You can safely combine multiple glyphs in a plot, even if some are rendered in WebGL, and some are not.

[2020-10-11] hmm, didn't seem to have any effect?? dashboard

TODO [B] [2020-10-11] FifthHour/correlator: Timeseries correlation in Python, Bokeh Server and on Heroku dashboardbokeh

TODO [B] [2020-11-01] JupyterLab Documentation — JupyterLab 2.3.0a1 documentation dashboard

TODO [B] [2020-11-11] raphaelvallat/pingouin: Statistical package in Python based on Pandas dashboard

STRT [B] [2020-10-31] How I built a spreadsheet app with Python to make data science easier | Hacker Noon dashboardspreadsheet

TODO [B] [2020-10-28] I just updated my Pandas GUI project to have some sample datasets, here it is working with a Simpsons dataset and proving that the early seasons are the best… /r/compsci dashboard

TODO [B] [2019-11-24] Jupyter tools to increase productivity - Towards Data Science dashboardjupyter

[2020-05-12] widgets, debugging, toc dashboardjupyter

TODO [2020-09-10] from IPython.core.debugger import settrace dashboardjupyter

TODO [B] need to post it somewhere… but not sure how to present to people. could def post some plots to quantifiedself? dashboardtoblogqs

CREATED: [2019-04-09]

TODO [C] [2019-04-09] Short- and long-term effects of a single bout of exercise on heart rate variability: comparison between constant and interval training exercises. - PubMed - NCBI dashboardhrv

https://www.ncbi.nlm.nih.gov/pubmed/15461995

R-R intervals, TP, and HF/TP were significantly decreased while LF/TP and LF/HF were significantly increased during the early recovery, when compared with control values. This could be a response to the significant decrease in SAP and DAP at this time. Twenty-four and 48 h after the end of the exercise, HRV parameters were at the same levels as before exercises in the supine posture, but a persistent tachycardia continued to be observed in the upright posture, together with reduced TP values, showing that cardiovascular functions were still disturbed. The short-term HRV recovery seemed dependent on the type of exercise, contrary to the long-term recovery.

TODO [C] [2019-08-17] Plotting with categorical data — seaborn 0.9.0 documentation dashboardviz

https://seaborn.pydata.org/tutorial/categorical.html#categorical-tutorial

Boxplots
The first is the familiar boxplot(). This kind of plot shows the three quartile values of the distribution along with extreme values. The “whiskers” extend to points that lie within 1.5 IQRs of the lower and upper quartile, and then observations that fall outside this range are displayed independently. This means that each value in the boxplot corresponds to an actual observation in the data.

[2020-09-02] not sure if boxplots in particular are usefult to me? dashboardviz

STRT [C] thinking about making it public dashboardhpi

CREATED: [2020-05-13]

where to run it? it's gotta be sufficiently dynamic?
run from google cloud? suck in the data once? or refresh continuously?
maybe run the original data retrieval? not sure

TODO [C] add more dashboards to web? dashboard

CREATED: [2018-10-30]

TODO [2019-04-10] could add to dashboard? dashboard

TODO [C] 'Other' workouts i'm using for HR decay – shit, I think I lost hr here after 10th minute. Could detect it automatically I guess? dashboardqshr

CREATED: [2019-08-18]

STRT [C] always keep a tab running on one for my desktops? or pinned dashboard

CREATED: [2020-07-25]

[C] [2019-04-06] shit. influxdb seems to be very unsuitable for the kind of thing I want. also very awkward to render dashboards… dashboardinfluxdb

[2021-02-13] hmm I guess what I meant is that it's for aggregate data only, and a bit awkwards to explore by-point data? dashboardinfluxdb

STRT [C] I really want proper and fast location history.. dashboardlocation

CREATED: [2020-07-28]

STRT [C] run with cachew? also integrate with hpi dashboardcachewnutrino

CREATED: [2020-08-06]

TODO [C] show my dashboard dashboardqs

CREATED: [2019-08-05]

[C] [2020-09-01] Pandas API — hvPlot 0.6.0 documentation dashboard

Lag Plot
Lag plots are used to check if a data set or time series is random. Random data should not exhibit any structure in the lag plot. Non-random structure implies that the underlying data are not random.

STRT [C] mixing in data from taplog dashboardweightexerciseqs

CREATED: [2020-10-13]

Dump an org table
Preserve the ids
Join with taplog (maybe even as a df)
Win! Maybe check some similarity score too
Same for weight?

TODO [C] on the hrv plot, diplay as arrow if the trend is up or down dashboardsleep

CREATED: [2020-11-17]

STRT [C] [2020-11-03] joshlk/dataclassframe: A container for dataclasses with multi-indexing and bulk operations. dashboard

hmm.. doesn't have any typing imports at all?? what do they mean 'working nicely with type hints', that they don't crash??

[2021-02-06] ah ok, something added.. maybe reevaluate it later dashboard

TODO [C] use for daily dashboard or something? dashboardremarkable

CREATED: [2020-12-13]

TODO [C] [2020-09-06] bokeh/notebookembed.ipynb at 2.2.1 · bokeh/bokeh dashboard

STRT [C] https://github.com/thesephist/histools dashboardviz

CREATED: [2020-12-24]
A collection of tools for generating data visualizations from browser history data

looks like a heatmap?

TODO [C] [2020-12-30] dashboardbokeh

goddamnit. so frustrating that I can't zoom it properly

STRT [D] Grafana dashboardtimeline

CREATED: [2018-01-31]

eh, I'm not even super sure what I wanna plot…
I guess for beginners would be nice to have sleep plot, maybe estimated from rescuetime
that could also href to timeline slices
plotly plugin
geomap plugin
https://forum.quantifiedself.com/t/grafana-influxdb-and-apple-watch-metrics/5323
apparently some people are using influxdb for that stuff? still not worth for me…
http://quantifiedself.com/2015/07/2015-qs-visualization-gallery-part-3/
https://github.com/Freeyourgadget/Gadgetbridge/issues/49

uhh. so unclear how to connect my data to grafana. dashboardtimeline

STRT lastfm dashboardtimeline

some issues: unable to hide zeros —> LOTS of 0 points
unable to attach labels?
influx to start terminal client
lastfm/fillinfluxdb.py

http://localhost:3000
influxdb on default port… everything is admin/admin

I guess, start with weight for simplicity dashboardtimeline

then, do sleep dashboardtimeline

TODO [D] figure out grafana.. dashboard

CREATED: [2019-02-20]

TODO [D] post on dataisbeautiful? dashboardredditsleep

CREATED: [2019-07-22]

TODO [D] fix dovpandas warnings dashboard

CREATED: [2020-01-04]

TODO [D] when I release it, make sure it works both against public and private hpi bits dashboardsleep

CREATED: [2020-08-07]

TODO [D] profile stuff a bit? dashboard

CREATED: [2020-10-18]

TODO [D] [2020-12-05] xkcd: Linear Regression dashboardtoblog

DONE [B] [2019-04-15] Python Data Visualization 2018: Why So Many Libraries? - Anaconda dashboard

TODO [2019-09-10] try using in dashboard dashboard

DONE [2019-04-14] irissplom.py — Bokeh 1.1.0 documentation dashboard

[2019-07-23] eh, defaults don't look great. maybe it's faster but unclear dashboard

CANCEL [B] [2020-10-25] Tableau Desktop dashboardvizqs

-----–— last housekeeping on [2021-02-06] ---------- dashboard

TODO [C] can overlay activity data vs sleep data from different tools? dashboardsleephpi

CREATED: [2021-02-12]

TODO [B] [2021-02-12] grafana/worldmap-panel: Worldmap panel plugin for Grafana 3.0 that can be overlaid with circles for data points. dashboardlocation

TODO [B] [2021-02-18] Personal Dashboards for Self-Tracking Data - Quantified Self / Apps & Tools - Quantified Self Forum dashboardhpipublish

some screenshots

TODO [B] could use mybinder for demos? same used by jupyterlab dashboardhpi

CREATED: [2021-04-11]

TODO [C] [2021-04-11] LibrePhotos/librephotos: Self hosted alternative to Google Photos dashboardphotosmemex

TODO [C] [2021-04-11] photoprism/photoprism: Personal Photo Management powered by Go and Google TensorFlow dashboardphotosmemex

looks good – I guess need to evaluate on the basis of whether I can share it?

TODO [B] [2021-03-05] Introducing Orion: A Powerful Substitute for OwnTracks Recorder - Blog - Kevin Lin dashboardlocationHPI

orion-web provides a fast, interactive, React-based data visualization frontend for the web, powered by deck.gl

whoa ok, this is pretty awesome

STRT [B] [2019-09-30] Jupyter notebooks on steroids dashboardipynb

TODO [2019-10-06] try ipywidgets? dashboardipynb

TODO [C] [2021-04-29] Personal Dashboards for Self-Tracking Data - Quantified Self / Apps & Tools - Quantified Self Forum dashboardqs

I pretty much 100% agree with your assessment. I personally would also add:

    Automatic backup of data.
    API for simple daily import of data (e.g. from manual or unsupported trackers). To me CSV import is inferior solution, especially for dashboard, where the point is to have the data visualized persistently.
    I would expand the ‘flexible data analysis’ to following specific analysis that I feel are essential (but I haven’t seen anywhere):
    a. cross-correlation across time to uncover temporally delayed relationships between tracked variables
    b. automatic application of the above to all pairs of variables with report of the strongest relationships found
    c. proper reporting of statistical significance of any correlations found
    d. multi-variate regression

motivaion: good points about what's missing from most current solutions

Jump to search, settings & sitemap