The authors emphasize their ability to perform action alignment of actions performed by different people at different times. Yet it seems like with their constraining the alignment to 1D affine transformations in time (ie different frame rates or time scales) their method is unlikely to scale well to longer sequences where non-linear temporal warping becomes more pronounced. Some of this warping is apparent in the ballet clip (I'm sure there would have been more if there wasn't music being danced to), and this is a fairly short clip. I wonder how quickly performance would drop off as the linear assumptions begin to fail.

The novel aspect to the paper is the way they solve the optimization problem or is it the way they construct the problem itseft and the optimization is just plugged from a standardized routine?

Also from the examples it seems like, they are able to handle the translation and temporal scale well, but they do not show an example where the object is heavily transformed(affine)? What do you think?

In response to Matt: My guess is that you could segment the sequence temporally and do alignments on sequences that could be linearly aligned if nonlinear temporal scaling was a nasty issue. As an example, say your ballerina fell twice but otherwise performed at full speed. If you segmented it properly the ground time would just be a VERY slow progression through the perfect performance with a low match metric. Just a thought mind you.

I may not be understanding the paper correctly, but could this paper be described as doing something analagous to RANSAC, except in space-time, rather than just space? Did that make sense?

My question is in the same vein as Nikhil. Some of techniques used to enhance convergence such as those described in "Confidence weighted regression" seem quite unique to the problem domain. I was wondering if these anyone had seen these particular techniques applied to other optimizatio problems.

The results look really cool, but I imagine this takes years to run even for short sequences. I brought this up when we were talking about the cuboids paper, but I wonder if a more randomized search like RANSAC could work sufficiently well and be a lot faster.

It'd be really interesting to see how well this performs in a behavior recognition framework. Specially when compared with cuboids, since cuboids performed better than Zelnik and Irani's "Event-Based Analysis of Video" which similarly recognized behavior regardless of differences in spatial textures such as clothes.

The results are impressive especially when fusing multiple sensors together for the same schene. I would be interested to see how wide the baseline between the sensors can be before this algorithm breaks down.

re Anton: This technique might be too specific in the behaviors it would recognize, apart from it being slow. I'd be more interested in seeing the cuboid approach applied to the sequence alignment. By taking into account relative position between the cuboids, and using something like RANSAC, as mentioned earlier, to try and find a transform that aligns matching cuboid features. I'd imagine this could be more robust and efficient.

Pretty cool paper. I also wonder if outliers can be detected in another way or complementary to the concavity constraint for the quadratic approximation of the normalized correlation function.

Another question is about the computation time/space required for comparing the two sequences, in the case they want to use this for action/event recognition or any other of the applications that they mention.

In response to Tom's response to me: That might work, though you'd have some interesting tradeoffs in window size.

It seems though that time is just fundamentally different in that it's ordering more than distance that matters. You see this a lot in speech recognition with Markov chains and such, where each phoneme can be of (almost) any length and it's the order of phonemenes that really matters.

ugg bags ugg bags ugg bags ugg bags ugg bags uggs