Category Theory, Entropy and Information Theory

Just for reference, but this looks awfully relevant..

n-Category Cafe May 10, 2011 's entry: Entropies vs. Means (Tom Leinster) as a start, which then takes us to Entropy as a functor and specifically A characterization of entropy in terms of information loss.

This takes us neatly to Reyni's paper A. Rényi, Measures of information and entropy, in Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability 1960, pp. 547–561.

The thesis being:

Given an ontology, we can calculate the entropy of a set of attributes/classes from that ontology, ie, the entropy of a function that selects a sub-set of that ontology. This is of course complicated by dependencies between attributes within that ontology, eg: star-sign <---> date-of-birth  etc.

For each sub-set we can construct a further set of functions that partition the ontology as above - thus creating a partial ordering of functions (and a semi-lattice where all functions can be ground to a bottom value by a function f O -> _|_ which effectively removes everything giving an entropy of 1, ie: pure randomness and loss of all information.

A privacy preserving function is one that introduces more entropy, ie: obfusicates or anonymised any data passing through. However, There are certain other properties that need to be investigated, such as aspects over the original ontology - not all information is PII etc. Does this imply some weighting in the entropy calculation or something more exotic such as a matrix structure.

This might fit in nicely with some earlier work on trajectories of information....