Importing Video and Stems / Offset?

Hi there!

I do video editing in Lightworks on kxStudio 18.04. Since Lightworks doesn’t really have audio features, I usually do stem export there, and then I do film mixing in Ardour. Luckily, I can add the video to my Ardour project.

However, the extracted audio from the video file as well as the video itself and the audio stem files are not in sync. When I open the stem files and the video file in Audacity (which will extract the audio from the video), they are perfectly in sync. After importing stems and video into Ardour, they are not. The stem tracks are too early.

Is this a bug, is there something I don’t understand?

Since quite some NLEs allow to move audio only in the unit of video frames, re-aligning stuff is always uncomfortable. It would be great to simply have stuff properly aligned right away.

Hints are welcome. :slight_smile:

Best
Niels

Just a thought, video sound is 48,0 khz, Ardour isn’t in 44,1 khz?

I always work in 48khz, so this is not the case.

What framerate does the video have? Is it a SMPTE standard that Ardour supports?

If not, enable:
Menu > Session > Properties > Sync > Use Video File’s FPS Instead of Timecode Value for Timeline and Video Monitor.

1 Like

Nah ardour being on the wrong sample rate would cause it to sound lower in pitch, and drift in sync over time, not start out of sync.

If it is starting out of sync my first thought might be that it has to do with the particular video codec you are using to export out of Lightworks. I haven’t used Lightworks in a bit, but when I did I did exactly what you are describing with no issue, but I remember I had to spend time finding the correct format to import into and out of Lightworks as it is a bit picky on Linux. It wouldn’t surprise me if the import process for Ardour doesn’t handle specific codecs with timing correctly.

Usually what I tried to do was to make sure my video editing was complete in lightworks, stem export with a HQ video, then import into Ardour and finish audio there, stem export from Ardour, and remux in ffmpeg. I could also go through and export straight from Ardour on occasion so long as I told it to always use the original video file. But either way my suspicion is the precise video codec you are using for export from Lightworks may be the issue combined with the precise ffmpeg settings Ardour uses.

        Seablade

EDIT: Also see Robin’s post:)

Mhhh. My videos usually are 25.000fps. So I did some more testing.

It turned out that the issue arises when I export MP4 from Lightworks. The audio extracted from the MP4 shows that audio and video are too late (nudged to the right).

When I go via AVI with CineForm in it, things turn out just right. However, my aspect ratio gets screwed up in the preview monitor, it gets interpreted as 4:3 with two huge black bars left and right. VLC plays it correctly, though.

So I tried MOV with DVCpro100 in it. This works properly. Correct aspect ratio and correct “timing”. Good for a start!

Like using CineForm, DVCpro100 has the drawback of creating very large files. And you cannot mux the audio back into the final video in Ardour, if desired to do so since you don’t have the final MP4 available. (I usually don’t do this, but others might want to. I usually import the exported audio mix back into Lightworks and then export a new final video.)

Have others observed this, too? Is it the MP4s only from Lightworks? I think they’re pretty much good at conforming to standards, so maybe it is a general problem.

I want to emphasized that when I import the WAV and the MP4 into Audacity (which extracts the AAC audio from MP4 then), the tracks are aligned perfectly. So the MP4 output of Lightworks is likely to be correct.

It is more complex than that. It is hard to say exactly what is or is not wrong without looking at it far more, and it is possible that nothing is ‘wrong’ persay but that there is a specific edge case that wasn’t properly defined in the spec that can be interpreted multiple ways. All we can say is that the interaction between Lightworks’ export and Ardour’s import (Which uses ffmpeg and is usually pretty good as a result) of these files is causing an issue.

For the record I have issues similar to this between much better ‘known’ programs as well, it isn’t uncommon sadly. It is amazing how many times I have to fix things in the live entertainment side of things when someone brings in a video and says ‘It plays fine here…’

    Seablade

Audio and video inside the mp4 don’t necessarily have the same start timecode, there might be an offset for either one. When you separate video and audio from a file like this you lose the offset value. It’s not uncommon for example DVD ripping to have this exact same problem if audio and video are separated from each other when processing. This may or may not explain the behavior you are seeing. Maybe Audacity supports mpeg presentation timestamps (PTS) and Ardour does not ? This is just a guess though, maybe someone of the devs might shed some light on this.

Good call, the video-tools (timeline and xjadeo) do honor PTS, but any extracted soundfile is always placed at the start of the video (same PTS as first video-frame). So probably an Ardour bug.

@DrNI can you post the output of ffprobe -i /path/to/the/video.mp4 that may help to track this down (show per Audio/Video stream offsets if any).

Also ffprobe -show_frames /the/file.mp4 (that returns a lot of output). Only the pkt_pts_time of the first [FRAME] media_type=video ... [/FRAME] and first [FRAME] media_type=audio ... [/FRAME] are interesting.