Timecoding a streamed video

This post is as much for me as for anyone who reads this. Its to help gather my thoughts and try to predict any bugs or logical problems that might arise.

In some previous posts i mentioned a timecode that will be put into the xml file along with several other properties that help describe the point in space and time.

That part is fairly easy, all that needs to be done is to send off a timecode right?
Well, its not quite as easy as it might seem. Sending it off is the easy part, but getting the timecode itself to refer to a specific time during the video playback is bit harder.

The reason its hard. In flash, as far as i know, you cant just use the video timecode as a variable. The only access to the timecode seems to be through the video caption component, which could possibly be used, but i'll have to look into it later.

Anyhow, even if i could get that to work, a problem with the video arises. When a video loads it sometimes skips, or takes a little extra time to load and start playing the rest.

To get the current time and allow the drawing to work in sync yet independent of the video, we need to first figure out exactly what time the video begins playing. This is because no matter how fast or slow the connection, the starting time of the video will likely never be the same.

The flash movie broadcasts a time in milliseconds that represents the amount of time that has elapsed since the swf was initiated. This lets us find the time offset of the movie, which is not always the same, due to different connection speeds.

For example, a test i just did compares time offsets (time from swf initiation to video ready state) between 3 different connection types.
Local: 620 MS average offset (0.62 seconds)
56K : 11016 MS average offset (11.016 seconds)
DSL : 2159 MS (2.159 seconds)
T1 : 797 MS (0.797 seconds)

So according to my results, if i started recording mouseEvents for drawing as soon as the movie starts, the playback of the drawing could be anywhere from .62 seconds to 11 seconds out of sync with the video (assuming the drawing was recorded locally and played back via 56K modem (which isnt likely to happen), but the out of sync drawing will still be noticeable.

Currently, what i plan to do is calculate the offset for the recorded drawing. Then when the timecode is created and sent to xml it will be flexible..

For example;
A user records their drawing with a 500 MS offset. The offset is recorded as soon as the video starts playing. Once a drawing is initiated is grabs the current time and subtracts the offset.
if the first point were drawn at 00:00:00:550 it would be recorded into xml as 00:00:00:050. Then on playback, if the playback had an offset of 850 MS, the xml would start playing at 00:00:00:900 instead of 00:00:00:050.

This is one way around it. One issue i can foresee is the video buffer time, which may also have to be calculated and preserved.