I do video editing in Lightworks on kxStudio 18.04. Since Lightworks doesn’t really have audio features, I usually do stem export there, and then I do film mixing in Ardour. Luckily, I can add the video to my Ardour project.
However, the extracted audio from the video file as well as the video itself and the audio stem files are not in sync. When I open the stem files and the video file in Audacity (which will extract the audio from the video), they are perfectly in sync. After importing stems and video into Ardour, they are not. The stem tracks are too early.
Is this a bug, is there something I don’t understand?
Since quite some NLEs allow to move audio only in the unit of video frames, re-aligning stuff is always uncomfortable. It would be great to simply have stuff properly aligned right away.
Nah ardour being on the wrong sample rate would cause it to sound lower in pitch, and drift in sync over time, not start out of sync.
If it is starting out of sync my first thought might be that it has to do with the particular video codec you are using to export out of Lightworks. I haven’t used Lightworks in a bit, but when I did I did exactly what you are describing with no issue, but I remember I had to spend time finding the correct format to import into and out of Lightworks as it is a bit picky on Linux. It wouldn’t surprise me if the import process for Ardour doesn’t handle specific codecs with timing correctly.
Usually what I tried to do was to make sure my video editing was complete in lightworks, stem export with a HQ video, then import into Ardour and finish audio there, stem export from Ardour, and remux in ffmpeg. I could also go through and export straight from Ardour on occasion so long as I told it to always use the original video file. But either way my suspicion is the precise video codec you are using for export from Lightworks may be the issue combined with the precise ffmpeg settings Ardour uses.
Mhhh. My videos usually are 25.000fps. So I did some more testing.
It turned out that the issue arises when I export MP4 from Lightworks. The audio extracted from the MP4 shows that audio and video are too late (nudged to the right).
When I go via AVI with CineForm in it, things turn out just right. However, my aspect ratio gets screwed up in the preview monitor, it gets interpreted as 4:3 with two huge black bars left and right. VLC plays it correctly, though.
So I tried MOV with DVCpro100 in it. This works properly. Correct aspect ratio and correct “timing”. Good for a start!
Like using CineForm, DVCpro100 has the drawback of creating very large files. And you cannot mux the audio back into the final video in Ardour, if desired to do so since you don’t have the final MP4 available. (I usually don’t do this, but others might want to. I usually import the exported audio mix back into Lightworks and then export a new final video.)
Have others observed this, too? Is it the MP4s only from Lightworks? I think they’re pretty much good at conforming to standards, so maybe it is a general problem.
I want to emphasized that when I import the WAV and the MP4 into Audacity (which extracts the AAC audio from MP4 then), the tracks are aligned perfectly. So the MP4 output of Lightworks is likely to be correct.
It is more complex than that. It is hard to say exactly what is or is not wrong without looking at it far more, and it is possible that nothing is ‘wrong’ persay but that there is a specific edge case that wasn’t properly defined in the spec that can be interpreted multiple ways. All we can say is that the interaction between Lightworks’ export and Ardour’s import (Which uses ffmpeg and is usually pretty good as a result) of these files is causing an issue.
For the record I have issues similar to this between much better ‘known’ programs as well, it isn’t uncommon sadly. It is amazing how many times I have to fix things in the live entertainment side of things when someone brings in a video and says ‘It plays fine here…’
Audio and video inside the mp4 don’t necessarily have the same start timecode, there might be an offset for either one. When you separate video and audio from a file like this you lose the offset value. It’s not uncommon for example DVD ripping to have this exact same problem if audio and video are separated from each other when processing. This may or may not explain the behavior you are seeing. Maybe Audacity supports mpeg presentation timestamps (PTS) and Ardour does not ? This is just a guess though, maybe someone of the devs might shed some light on this.
Good call, the video-tools (timeline and xjadeo) do honor PTS, but any extracted soundfile is always placed at the start of the video (same PTS as first video-frame). So probably an Ardour bug.
@DrNI can you post the output of ffprobe -i /path/to/the/video.mp4 that may help to track this down (show per Audio/Video stream offsets if any).
Also ffprobe -show_frames /the/file.mp4 (that returns a lot of output). Only the pkt_pts_time of the first [FRAME] media_type=video ... [/FRAME] and first [FRAME] media_type=audio ... [/FRAME] are interesting.
OK guys… strange things. I tried to replicate the issue in a small example for this discussion here and now the MP4 audio has an offset when imported into Audacity, too. Most likely, Audacity also does ffmpeg stuff? Or the audio is really not on time for the MP4 export? Which would be a shame for Lightworks. I will annoy them a bit in their forum.
Thanks I got the files.
Quick look reveals that in the ffprobe dump, both the first audio and first video frame have a PTS of zero.
It really seems like an audio encoding issue inside the mp4.
The link doesn’t work for me. It redirects the Lightworks front-page.
–
Might be worth trying this with some more scientific approach: a beep in sync with a visual queue:
That one is encoded with ffmpeg, and was generated with the
