⭐ Backup cleanser

started compressing reddit…

• do not remove; move to killzone
  
• get all tables
  
• make sure all schemas match
  
• maybe convert it to json or something? and then compare jsons…
  
• checks that entries are dominated?
  

could name like this…

maybe before comparison explicitly 'cleanup' stuff

could use for processing… maybe via option

might be useful as a sanity check? to ensure stuff isn't deleted by accident? (like foreign key triggers)
e.g.

• run extract query first to get a snapshot of data
  
• run cleanup query
  
• run extract query first to ensure the data we care about is there?
  

shell globing nice for not typing too much

python3 -m bleanser.modules.firefox diff /path/to/database-201906{16,17}*.sqlite | less

that way might work more reliably… hmm

• append only (e.g. foursquare, hypothesis)
  
• rolling (e.g. rescuetime, github, reddit)
  

either way you can think of it as as set of strings

maybe keep all dirs that were there before – and only remove new empty dirs?

• e.g. 'all files are same'
  
• only added data
  
• rolling data (some fake datetime stuff with 30d retention)
  
• error in cleaner script
  

 return pipe(
     '.tags  |= .',
     '.posts |= map({href, description, time, tags})', # TODO maybe just delete hash?
     '.notes |= {notes: .notes | map({id, title, updated_at}), count}',  # TODO hhmm, it keeps length but not content?? odd.
)

e.g. on foursquare with isMayor: true – hmmm