How to Implement CINE in a web-browser

CINE isn't quite the same as a movie, in specific, there are a few areas that are different enough to warrant different handling:

CINE frames typically have full-resolution frames that they are associated with, even if the CINE is represented as MPEG or multi-frame MNG/GIF etc.
CINE frames can have markup, window levelling, etc applied to them - whether or not this should be turned off is another questions (perhaps turned off by default, but could be turned back on?)
CINE may have customized options such as frame subtraction, shutter (really, this is just markup) etc - this maybe customized by the user and not yet saved, so the CINE needs to be displayable in the same conversational state as the regular display.
CINE can just a fast way to navigate, OR it can be critical for getting exact timing, depending on whether this is CT type CINE or Cardiology type CINE (cardio work depends on fairly exact frame rates).
CINE could consume a huge amount of memory - eg a 1 hour US loop could consume gigabytes of space - it probably isn't reasonable to have all the frames in-memory and ready all at once, even if they are all locally cached. Even then, mostly the CINE display will be for a sub-range of frames.
CINE needs to be frame-rate or time-rate controllable much farther than movies, down to the 1 frame per second and upwards of 200-300 fps (not all of these would be shown, some form of frame skipping would be required.)

An ideal solution:

Reads images ahead of time, probably on a few threads to allow resources to be fully utilized
Dynamically selects resolution and display quality
Doesn't use too much memory, eg by holding image hrefs to all the images once they have been shown once
Only performs the WADO requests during playback - except if the user performs another action (eg changing the frame positions, rate, or editing marking elsewhere etc)
Can use MPEG or another multi-frame format if available, otherwise falling back to single-image display or dynamic conversion
Has enough of a look-ahead cache to allow for retries etc - probably 2 s of look ahead is required
Extensible to allow synchronized multi-series display, including ECG synchronized display

Thoughts on the design:

Have a set of "actions" that need to be done, and be able to add to the list
Implement some sort of marker than can indicate time-delay issues
Implement a display timer that attempts to display at a given rate - at that time, display the nearest frame available
Have a strategy to read ahead some, and then fill-in time details
Have the XML transformations ready well ahead of time, but be able to request new ones if the transform keeps on running too long (eg for hour long CINE playback, it will need to make multiple server requests) - probably 1-2 seconds ahead of time, as this is a multi-step process.
Always "playback" at a time rate/position with ID, rather than at a specific frame position - that allows multi-frame synchronization by default.
Throw away frames as needed to keep the cache size reasonable - may keep some high level frames
Integrate with fast-movement handling, and CT/MR linking so that the same cache is useable (that is, if fast frame navigation is being done, it might re-use existing known frames, rather than displaying random frames - then, load local frames into a cache when navigation slows.)

What are the use cases to consider?

Cardio - CINE playback of relatively short segment looking for exact timing information - eg want 20-30 fps at a specific rate displaying every frame within 20 or so ms of actual time, and indicating whenever that fails.
CINE to display overall area - individual images matter, want to see a selection of them in a continuous fashion so that the user doesn't need to go through everything by hand
1. Stack based - go from image N to N+M in order, possibly skipping for performance reasons
2. Slide location - change distance within the frame of the slide, eg pathology panning smoothly across the slide,
3. Time/Location based - from from one position in Time+Position to another, possibly synchronize on time OR location OR both (eg ECG synchronizes to time, while a current/prior MR synchronize on position, with custom user modifications to the position mapping.)
CINE to load things into memory, so that later navigation is fast.
At end of loop, options are:
1. Reset to beginning of loop
2. Reverse the display
3. Continue till ALL segments complete the update in that "direction"
4. Apply one or more time/space/rotation/location vectors and continue (eg pan over slide in x, then add +y and reset x, pan in x, etc, or CINE in echo 1 of MR, then echo 2, then echo 3, then repeat).