Hey #fediverse, could you help me out here?
I'm working on some experiments based on two assumptions:
- Some information has decaying value;
- The amount of information that has decaying value is increasing rapidly.
It's somewhat early stage, so before I go down a rabbit hole, does anyone know of any prior art, or any articles or algos on a related topic?
Thanks!
@kellerfuchs Initially, filtering. "Should I be considering this at all, and if so, in what weight?"
Trivializing it, you could say I'm looking at a more elaborate exponential moving average, although used to calibrate age rankings more than adjusting a datum.
@ricardojmendez
OK. I guess this is time-series data?
If so, https://blog.filippo.io/understanding-metrics/ is a pretty decent intro to the various kinds of time-series data, and while it argues that approximate, consise representations are not needed (“keeping all requests timings of the last 5 seconds to nanosecond precision at 5000qps [requires] 200kB”), it refers to various techniques (weighted- and sliding-window-reservoirs) that are relevant.
[continued]
@ricardojmendez
Searching for “consise approximations of time-series” yielded “Piecewise Trend Approximation: A Ratio-Based Time Series Representation” [0], which itself refers to a bunch of different methods in the introduction.
That is definitely a thorough bibliographic survey, but it is a start; someone more expert than I might pitch in, or you might find better sources, though.
@kellerfuchs Thanks for the links. I hadn't thought about it as a time series, given that the creation timestamp is only a component and the associated datum is not necessarily quantifiable, but it's a good starting point. I expect discounting old data is relevant in time series, might lead to good pointers.
Outside perspective ftw.
Cheers!
@ricardojmendez Ah, yes, if the data isn't quantitative it might be much harder.
Good luck :)
@ricardojmendez #history historians have been dealing with this for as long as we've had them ...
@ricardojmendez my lovely history-major gf is sending relevant links ...
@ricardojmendez
http://museums.in1touch.org/uploaded/web/docs/Documents/CMA%20Deaccessioning%20Guidelines.pdf
http://www.robertjweisberg.com/metadata-and-the-museum-digital-dilemma/
http://www.tandfonline.com/doi/abs/10.1080/01639374.2016.1192078?journalCode=wccq20
https://journals.ala.org/index.php/ltr/article/view/4690/5581
Deaccessioning seems to be the keyword here.
@ricardojmendez some of those even have flowcharts...there's your algos
@jeffcliff Excellent! Please thank your lovely history-major girlfriend on behalf of a grateful internet person.
@ricardojmendez
Those assumptions makes sense, sorta, though you didn't clarify which sort of value you care about.
However, it's so vague and broad that it's really hard to comment on (or refer you to existing work).
What's the question your experiments try to answer? What sort of algorithms/experiments/data do you have in mind?