In response to Tom's response to me: That might work, though you'd have some interesting tradeoffs in window size.

It seems though that time is just fundamentally different in that it's ordering more than distance that matters. You see this a lot in speech recognition with Markov chains and such, where each phoneme can be of (almost) any length and it's the order of phonemenes that really matters.