-
Notifications
You must be signed in to change notification settings - Fork 2.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Seeking to end of previous fragment results in media error, or wrong frame being displayed #2327
Comments
Do you need any further information about this issue? |
I think there is enough information here. Thank you for the great report and test page! |
Hi @MsLizzie, This appears to be fixed in the latest release. Can you please confirm? https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html If you're able to reproduce in latest and I missed something I'll reopen the issue. Thank you! |
Hi Rob, Unfortunately, this issue is not resolved. I've checked everything again, using the latest versions of browsers with the same test page: The test page uses the latest version: My browser versions: I've checked using 2 Macs, and 1 PC as follows: Chrome (Mac OS High Sierra) Note that as before, the issue seems to relate to seeking near the end of a segment when the video has not been cached yet. On all browsers checked, if you refresh the page, then press the buttons in this order: Button 3 -> Button 2 -> Button 1, then the results are perfect, and the correct frame is seeked to on all browsers. However, the following does NOT work: Test: Refresh page. Click button 1 (Seek 55676). After the seek completes click button 2 (Seek 55436) Results: So as you can see above, all browsers have failed the test, either because they display the wrong frame in the 2nd seek, or nothing happens for the second seek. Note also that the secondary minor bug also still occurs in Safari: Thanks for your help Liz |
Hi @MsLizzie, Frame accuracy is a browser issue (sometimes an HLS packaging issue). It's not something hls.js can guarantee. As far as why it is not displaying the correct frame, I have an explanation below. For clarity, let's list the issues as it's tricky for me to understand what the main issue is here with all the information provided. Thanks for being patient as I catchup. Issues:
I would ask that you try the demo Timeline to troubleshoot the issue. It should help illustrate where browsers actually seek, as well as where hls.js plots each segment according to the media it parses. The player is seeking, but hls.js is not buffering media at the point it seeks to, and because the video is paused the video does not update. hls.js "thinks" the segment ahead of the time you are seeking to is the correct segment to buffer. To fix this you should try reducing |
Hi Rob, I will take a look at the demo (with timeline), Although the only case I really care about at the moment is when the video is in pause mode, and you perform a seek. The problems occur only when the seek is near the end of a segment, so I'm not sure how I can duplicate this easily in your demo, as it doesn't seem to allow you to seek to precise locations. FYI: For us, hls-js generally works very well, and we do rely on frame accurate seeking. Before we used hls-js, the frame accurate seeking worked perfectly in all browsers on single unsegmented video files in all cases frame rates, etc. With hls.js, this is almost true, and only fails in the case where seeks are very close to the end of a segment that hasn't been cached yet. This is a very exceptional case, and it was hard for me to reproduce reliably (in our software 99%+ of random seeks worked perfectly), which is why I created my example test page when I managed to find specific seek locations that would cause problems consistently. So it seems to me that this issue is all about hls-js overshooting the required segment. You may not be able to guarantee frame accurate seeking, but I believe that will become the outcome as far as I am concerned if this issue can be fixed. You said, though, that setting maxFragLookUpTolerance to zero will ensure that the correct segment is loaded. maxFragLookUpTolerance: 0.0 So with a maxFragLookUpTolerance of zero there are still problems seeking near the end of segments. Thanks Liz |
We've done @MsLizzie's test (using the exact same stream and same seek points) with video.js (current released version) and gotten perfectly frame accurate seeking results in all browsers with that, so I'm not sure just dismissing that part as a browser issue is the right answer. |
Hi Liz, The screen shot above is the result of me seeking in the dev tools exactly as your test page does without setting Outside of that is there any other issue? If the browser sets currentTime accurately AND media is buffered at the time presented, then is the wrong frame still rendered? |
Hi Rob, Once the media has been buffered, the seeking works perfectly across all browsers. The issue I am having is specifically when we seek to a location that hasn't been buffered yet. I'm struggling to see the workaround. The problem I am seeing has always occurred when the maxFragLookupTolerance is set to zero already. It is important to seek to exactly the locations I specified to demonstrate this, which I have now done with your demo (https://hls-js.netlify.app/demo/) as follows, by performing the seeks from the javascript console: I changed the config in your demo player to use the same as mine, although I don't think it makes any difference. { Stream: Make sure the video is paused as quickly as possible. So the primary issue I have is demonstrated by following these steps: Chrome, from JS console: Firefox, from JS console: I've tried different combinations of configs, with different values of maxFragLookUpTolerance, but I cannot get your demo to work correctly in either Chrome or Firefox in this case. Everything behaves exactly the same way as my own test. Hope this helps! Thanks Liz |
As a follow up to the above, I want to mention the following again, in case it was lost in the above.... In the example above, a seek to 2319.84 seconds works, but when followed by a seek to 2309.84 seconds it always fails (when the video at 2309.84 isn't cached yet) Whereas, if the video is loaded again, and you seek straight to 2309.84 it works fine (all browsers) One difference is that in the first (fail) case the seek is a backwards seek from 2319.84 to 2309.84, but in the second (success) case, it is a forwards seek (from 00:00:00) I have only ever seen the failure (so far) when you seek backwards in the video, and the destination location is not cached yet. Could this be relevant to the problem? Thanks Liz |
Yes. The issue is that the segment needed to display the frame is not buffered when the playhead only passes (close to) the end of the segment, and the next segment is buffered. I see that with This is an issue in the stream controller's So, when seeking to a position near the end of a segment:
Set |
Hi, thanks for the information. I am still experiencing issues, however. Has something changed in your demo player since last week? It's now behaving differently than before: Considering again Seek 1: 2319.84 seconds followed by Seek 2: 2309.84 seconds (uncached) Old behaviour: Today's new behaviour: This new behaviour occurs whether you set maxBufferHole to zero or not. However, today I am seeing a divergence between my player test page and your demo player for the first time: https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html When I set maxBufferHole to zero, there IS an improvement on my test page. However it does NOT fix the Firefox issue, and it still seeks to the wrong location, as before, Why is there now a discrepancy between what I see in my test page and your player. Your player is now working worse, for some reason. Thanks Liz |
I am unable to reproduce the media error in v1.0.0. If there is a specific environment that produces a media error in v1.0.0 please log a new issue focused on creating that error and we can prioritize that. For the seek issues, I can perform the two seeks above in Chrome and Firefox with Chrome/Firefox: 1st seek to 2319.84 seeks to frame 55676 This does not appear to be an issue hls.js can resolve as Firefox is determining to render the nearest available frame and completing the seek before hls.js can load and append the media. The fact that you can do a follow-up seek is promising if you are in need of a workaround. You might consider adding a listener for If you feel that there is a change hls.js can make to fix a certain type of behavior please file a new issue with a narrower area of focus. I apologize for not being able to help more generally with frame accuracy across browsers, but the scope of the issue makes it difficult to suss out what is a browser issue, what we can workaround internally or you might in your implementation, and what is a critical error most end users would notice that we should prioritize over everything else. |
Starting with Lizzie's test link - https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html - using latest Firefox on Windows. Seek to 55436, shows 55436, then seek to 55196 and it shows 55200, then seek to 55196 again and it shows 55196. If this is really a browser issue, then it should show up regardless of player used, so I quickly mocked up the same sample in video.js https://snapstream-dev-test.s3.amazonaws.com/hls_seek_issue_with_videojs_7.10.0.html https://snapstream-dev-test.s3.amazonaws.com/hls_seek_issue_with_videojs_current.html With the 7.10.0 url, you'll see that in FireFox you can seek to 55436, shows 55436, then seek to 55196 and it shows 55196, seek again and it still shows 55196. With the current url, you'll see that in Firefox you can seek to 55436, shows 55436, then seek to 55196, and it shows 55201, seek again and it shows 55196. Three different behaviors depending on the javascript library used (including one that works perfectly) lead me to believe it is not a browser issue. |
Hi @SnapStreamJason ,
videojs 7.10.0 clears the buffer whenever seeking outside of the buffered range. I confirmed this by checking If this is the behavior you want with hls.js, you can flush the buffer on video "seeking" using |
#4956 has brought up the topic of seeking back frame-by-frame. As long as |
Hi! Thank you for your great tool. I'm not sure I have to create a separate issue for my problem. I've got similar issue with seeking stuck after a few seeks to same time 122.82. Reproduced in both chrome and firefox on mac, on latest hls.js and dev. here is the link to demo with the stream and here is the config:
To reproduce you can just run the script in the console
after few seeks it got stuck |
What version of Hls.js are you using?
The latest version:
https://cdn.jsdelivr.net/npm/hls.js@latest
What browser and OS are you using?
Safari / Chrome / Firefox, Windows 10 / Mac High Sierra. Exact symptoms are browser specific.
Test stream:
https://d25uzy1v1dityp.cloudfront.net/master_playlist.m3u8
Test page, demonstrating issue
https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html
Checklist
*** The issue can't be easily demonstrated in the demo due to the need for specific configuration, and the ability to seek to specific frames, although the stream will play. Please use my test page hls_test.html to see the issue. Instructions on how to use follow.
*** The issue can't be easily demonstrated in the demo due to the need for specific configuration, and the ability to seek to specific frames, although the stream will play. Please use my test page hls_test.html to see the issue. Instructions on how to use follow.
General description of issue
When seeking from fragment X to a point near the end of fragment X - 1, the correct frame is not always displayed. The actual result is browser specific, and not 100% consistent.
My test page (https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html) can be used to demonstrate the problem easily.
The test page (https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html)
The test page contains 3 buttons, allowing seeking to 3 specific frames in the video.
Each frame of the video has burned-in time code to make it easy to check which frame is being displayed.
If the page is reloaded before pressing each button, then the correct frame is seeked to in ALL browsers.
To ensure that this works correctly, 2 changes to the configuration were required.
The problem only manifests itself when the seek buttons are pressed in sequence (left-to-right) without refreshing the page.
The first seek button seeks to 3 frames before the end of fragment 231
The second seek button seeks to 3 frames before the end of fragment 230
The thrid seek button seeks to 3 frames before the end of fragment 229
Steps to reproduce
Use my test page to reproduce the issue (https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html)
Press each of the seek buttons in sequence (left-to-right). Several results are possible (different results seems dependent on the amount of time you wait before pressing the 2nd and 3rd seek buttons). Refresh the page running each test below.
Firefox
Test 1) Press "Seek 55676" (which will correctly display frame 55676) then wait (say) 8 seconds, then press "Seek 55436". This results in a media error (indicated in the browser console), although frame 55436 is displayed correctly once hls.recoverMediaError() completes.
Test 2) Press "Seek 55676". Once frame 55676 is displayed, immediately press "Seek 55436". This time there is no media error, however the wrong frame is displayed (55440)
Test 3) Press "Seek 55676", wait 8 seconds, press "Seek 55436" wait 8 seconds, press "Seek 55196". There is no media error, but the wrong frame (552200) is displayed.
The behaviour is identical on PC (Windows 10) and Mac (High Sierra)
Chrome
Test 1) Press "Seek 55676" (displays correct frame 55676), wait 8 seconds, press "Seek 55436". This results in a media error. Frame 55436 is displayed correctly once hls.recoverMediaError() completes.
Test 2) Press "Seek 55676". Once frame 55676 is displayed, immediately press "Seek 55436". After about 2-3 seconds, there will be a media error, however hls.recoverMediaError() doesn't manage to recover, and the video is completely white. There are no additional errors in the console. At this point, the video is no longer playable, and the page needs to be reloaded.
Note that the behaviour described in this test occurs roughly 80% of the time on a Mac, and about 20% on a PC. The rest of the time, the video recovers, and the correct frame gets displayed.
This behaviour is reproducable on PC and Mac.
Safari (Mac)
Test 1) Press "Seek 55676". Displays frame 6 briefly, before correctly displaying 553676. This is just noted in case it has any relevance to the main bug I am submitting here.
Test 2) Press "Seek 55676" then press Seek 55436. This results in a media error irresepective of the delay between pressing the buttons. Frame 55436 is displayed correctly when recoverMediaError() completes.
Test 3) Press "Seek 55676", then "Seek 55436", then "Seek 55196". You will get a media error on both of the last 2 seeks. This seems 100% consistent on Safari, and recoverMediaError() always seems to work.
Expected behavior
When seeking to a specific location, the frame displayed should be consistent, and not dependent on previous seeks in the video. All browsers should behave in an identical way, and the seek should not result in a fatal media error (Hls.ErrorTypes.MEDIA_ERROR)
Actual behavior
Variable. Described in detail in the steps to reproduce, above.
Either a media error occurs (which may/may not be recoverable), or the wrong frame might be displayed (e.g. in Firefox)
Other comments
This problem appears to relate to caching. In the case where you seek to a point in the previous fragment where the previous segment hasn't been cached yet, then the fragment doesn't always get downloaded, resulting in a media error. In the case that the section of video has already been played through (and therefore cached), then the seeking works perfectly in all browsers.
Media errors in Chrome
When a media error occurs in Chrome, 2 entries are added in media-internals
Example below:
Entry 1
Player 650:96 (WEBMEDIAPLAYER_DESTROYED)
Player properties
audio_buffering_state BUFFERING_HAVE_NOTHING
event WEBMEDIAPLAYER_DESTROYED
pipeline_state kStopped
player_id 77
render_id 650
seek_target 2309.84375
video_buffering_state BUFFERING_HAVE_NOTHING
Log
00:00:00.000 seek_target 2309.84375
00:00:00.000 pipeline_state kSeeking
00:00:00.000 audio_buffering_state BUFFERING_HAVE_NOTHING
00:00:00.000 video_buffering_state BUFFERING_HAVE_NOTHING
00:00:00.001 event WEBMEDIAPLAYER_DESTROYED
00:00:00.001 pipeline_state kStopping
00:00:00.002 pipeline_state
Entry 2
Player properties
duration 2538.499999
event SUSPENDED
for_suspended_start false
found_video_stream true
frame_title
frame_url https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html
height 288
info Selected MojoVideoDecoder for video decoding, config: codec: h264, format: PIXEL_FORMAT_I420, profile: h264 baseline, coded size: [512,288], visible rect: [0,0,512,288], natural size: [512,288], has extra data: false, encryption scheme: Unencrypted, rotation: 0°, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}
is_platform_video_decoder true
origin_url https://d25uzy1v1dityp.cloudfront.net/
pipeline_buffering_state BUFFERING_HAVE_ENOUGH
pipeline_state kSuspended
player_id 87
render_id 650
seek_target 2309.84375
surface_layer_mode kAlways
url blob:https://d25uzy1v1dityp.cloudfront.net/7ced3796-9c2f-414e-9511-ac7b925daee5
video_buffering_state BUFFERING_HAVE_ENOUGH
video_codec_name h264
video_dds false
video_decoder MojoVideoDecoder
width 512
Log
00:00:00.000 origin_url https://d25uzy1v1dityp.cloudfront.net/
00:00:00.000 frame_url https://d25uzy1v1dityp.cloudfront.net/hls_seek_issue.html
00:00:00.000 frame_title
00:00:00.000 surface_layer_mode kAlways
00:00:00.000 url blob:https://d25uzy1v1dityp.cloudfront.net/7ced3796-9c2f-414e-9511-ac7b925daee5
00:00:00.000 info ChunkDemuxer: buffering by PTS
00:00:00.000 pipeline_state kStarting
00:00:00.036 found_video_stream true
00:00:00.036 video_codec_name h264
00:00:00.038 video_dds false
00:00:00.038 video_decoder MojoVideoDecoder
00:00:00.038 is_platform_video_decoder true
00:00:00.038 info Selected MojoVideoDecoder for video decoding, config: codec: h264, format: PIXEL_FORMAT_I420, profile: h264 baseline, coded size: [512,288], visible rect: [0,0,512,288], natural size: [512,288], has extra data: false, encryption scheme: Unencrypted, rotation: 0°, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}
00:00:00.038 pipeline_state kPlaying
00:00:00.041 seek_target 2309.84375
00:00:00.041 pipeline_state kSeeking
00:00:00.041 pipeline_state kPlaying
00:00:00.081 height 288
00:00:00.081 width 512
00:00:00.085 video_buffering_state BUFFERING_HAVE_ENOUGH
00:00:00.085 for_suspended_start false
00:00:00.085 pipeline_buffering_state BUFFERING_HAVE_ENOUGH
00:00:00.392 duration 2538.499999
00:00:16.131 pipeline_state kSuspending
00:00:16.131 pipeline_state kSuspended
00:00:16.132 event SUSPENDED
The text was updated successfully, but these errors were encountered: