Noob Q: How do I process sound in an existing video (of a musical performance)?

I have a VHS tape that I’ll have converted to most likely MP4 format. I’ve tried opening MP4s into Ardour and I only get the video, don’t see tracks or hear sound. I choose to run JACK/Alsa, Open Video Monitor Window (xjadeo) and Adjust Session Framerate to Match Video Framerate, Import/Transcode Video to Session, tried both Do Not Extract Audio and aac AAC etc, and I never see nor hear the audio tracks. I’ve been using Ardour quite happily to “remaster” previously recorded/mixed down music, but never worked with Ardour on video before.
What am I not doing/doing wrong? Any help would be appreciated.
FTR: Ubuntu Studio 20.04.4 LTS, Ardour 5, Thinkpad X230

Thanks,
realself

First try with Ardour6.

Second what does the output of ffprobe on that video say?

Seablade

"Extract audio : … " should have done the trick.

Could you check Ardour’s Log Window and/or enable “Debug Mode: Print ffmpeg command and output to stdout” in the Video Import Window? That might shed some light.

As an interim solution, you could extract the soundtrack manually as .wav file and then import this. something like ffmpeg -i the_video_file.mp4 soundtrack.wav

I think Ardour 5.x couldn’t import mp3 or any of these other Fraunhofer formats, so maybe Seablade’s comment should be followed. What Robin said should work as well, since .wav was never a problem…

While you aren’t incorrect about mp3 that wouldn’t be the cause in this case as for video imports the audio got extracted as .wav iirc

Seablade, thanks for your thoughts, sorry to take so long getting back. I’ve given enough money to Ardour already I don’t want to throw down another 45 just yet. Here’s ffprobe info on my test video:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ‘/home/kalidas/Desktop/60-Second_Karma_Wash-V2.mp4’:
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Duration: 00:00:45.44, start: 0.000000, bitrate: 1852 kb/s
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 1721 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/16/2019.
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/16/2019.

Robin, thanks for your help, the bash script you gave me worked! If I can’t get the extract to work I now have a fallback. Hope the sync would be good for an hour-long video…
Here’s some of the output from the Ardour log (not all of it, it’s 77 lines, mostly repeats of this):
The log for the session(s) kept saying Jack could not connect to server. The whole thing is 77 lines, here’s a sample:

[ERROR]: HTTP request failed: (7) Failed to connect to 127.0.0.1 port 1554: Connection refused
[ERROR]: HTTP request failed: (7) Failed to connect to 127.0.0.1 port 1554: Connection refused
[ERROR]: No LTC detected, video will not be aligned.
[ERROR]: HTTP request failed: (7) Failed to connect to 127.0.0.1 port 1554: Connection refused
[ERROR]: HTTP request failed: (7) Failed to connect to 127.0.0.1 port 1554: Connection refused
[ERROR]: JACK: Cannot connect to server socket err = No such file or directory
[ERROR]: JACK: Cannot connect to server request channel
[ERROR]: JACK: jack server is not running or cannot be started
[ERROR]: JACK: JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
[ERROR]: JACK: JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
[ERROR]: JACK: Cannot connect to server socket err = No such file or directory
[ERROR]: JACK: Cannot connect to server request channel
[ERROR]: JACK: jack server is not running or cannot be started
[ERROR]: JACK: JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
[ERROR]: JACK: Cannot connect to server socket err = No such file or directory
[ERROR]: JACK: Cannot connect to server request channel
[ERROR]: JACK: jack server is not running or cannot be started
[ERROR]: JACK: jack server is not running or cannot be started
[ERROR]: JACK: Cannot connect to server socket err = No such file or directory
[ERROR]: JACK: Cannot connect to server request channel
[INFO]: Loaded custom session defaults.
[ERROR]: No LTC detected, video will not be aligned.

Thanks again for your help.

Well a couple of things here… One you can test it with the demo version and see if it works there, and two you can choose to pay less for the compiled version of Ardour if you would like. Take a close look at that page when you download it.

But that being said, yes the command that Robin gave you is pretty much what happens on import with video in terms of the audio anyways (There is a little more to it IIRC but not needed to do it separately) and will allow you to work, in fact this was how I worked with audio from video for a long time before Robin’s fantastic work has led to the video timeline being where it is today.

As Robin suggested, putting up the debug output so that we can see the command run and the output from it (Just surround it in CODE tags here for readability) as what you posted doesn’t actually show the ffmpeg output would at least let us see what is failing, but I would test with at least the demo version of A6 to see if this isn’t something that has already been fixed.

   Seablade

Aha! So when importing the video, in the “open video” dialog, have you enabled
Extract LTC from audio and align video
?

If so that would explain why the soundtrack was not added to the session.

When enabled the audio-file is assumed to contain machine-readable timecode information for the video.

(LTC is a way to encode timecode and clock on an audio-track. It is commonly used in production to synchronize cameras and external audio recorders.)

1 Like

I’ve been doing it all: enabling Debug Mode: Print ffmpeg command and output to stdout; but I can’t find the stdout file. I also Extract TLC etc., and it says no TLC detected. I don’t know where the stdout print will be found, is it sent to the terminal? If so how would I see it there?

Disable “Extract LTC…”

as for the debug info, yes, you have to run Ardour from a terminal window to see it.

You could try to drag and drop video files onto Audacity and then export as an MP3 or WAV file to see what happens.

This is what Robin’s suggestion above does, it extracts the audio as a WAV file using ffmpeg.

Seablade

Seablade, I opened Ardour5 from the terminal, ticked all the boxes, hopefully I got you the printout you were asking for, here it is:
ffmpeg: ‘Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ‘/home/kalidas/Desktop/All One1/ALLOTHERS/New Spirit/QMVs/60-Second_Karma_Wash-V2.mp4’:
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Duration: 00:00:45.44, start: 0.000000, bitrate: 1852 kb/s
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 1721 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/16/2019.
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/16/2019.
Stream mapping:
Stream #0:0#0:0 (h264 (native) → mjpeg (native))
Press [q] to stop, [?] for help’
ffmpeg: ‘[swscaler @ 0x557f703d5740] deprecated pixel format used, make sure you did set range correctly’
ffmpeg: ‘Output #0, avi, to ‘/home/kalidas/karmawash15/interchange/karmawash15/videofiles/60-Second_Karma_Wash-V2.avi’:
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
ISFT : Lavf58.29.100
Stream #0:0(und): Video: mjpeg (MJPG / 0x47504A4D), yuvj420p(pc), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 10000 kb/s, 25 fps, 25 tbn, 25 tbc (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/16/2019.
encoder : Lavc58.54.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/10000000 buffer size: 0 vbv_delay: -1’
EXTRACT AUDIO:
/usr/bin/ffmpeg_harvid -i /home/kalidas/Desktop/All One1/ALLOTHERS/New Spirit/QMVs/60-Second_Karma_Wash-V2.mp4 -ac 2 -map 0:1 -vn -acodec pcm_f32le -y /home/kalidas/karmawash15/interchange/karmawash15/videofiles/60-Second_Karma_Wash-V2.avi.wav (null) (null)
actually writing state to /home/kalidas/karmawash15/karmawash15.tmp
renaming state to /home/kalidas/karmawash15/karmawash15.pending
actually writing state to /home/kalidas/karmawash15/karmawash15.tmp
renaming state to /home/kalidas/karmawash15/karmawash15.ardour
Butler drops pool trash
Found 2 along /home/kalidas/.config/ardour5/templates:/usr/share/ardour5/templates
run dialog

looks like this failed for some reason.

Coould you please manually run the command?

/usr/bin/ffmpeg_harvid -i /home/kalidas/Desktop/All One1/ALLOTHERS/New Spirit/QMVs/60-Second_Karma_Wash-V2.mp4 -ac 2 -map 0:1 -vn -acodec pcm_f32le /tmp/test.wav

PS again there was no soundtrack or sound, “Butler drops pool trash” I suppose refers to me closing the session (as there was nothing else to do).
Tried the demo of Ardour6.9 (though not from terminal), same results. But once Ubuntu Studio auto-upgrades I think I’ll buy the non-support version, perhaps both upgrades together fix the issue.

Is this what you’re looking for?
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ‘/home/kalidas/Desktop/All_One1/ALLOTHERS/New_Spirit/QMVs/60-Second_Karma_Wash-V2.mp4’:
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Duration: 00:00:45.44, start: 0.000000, bitrate: 1852 kb/s
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 1721 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/16/2019.
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/16/2019.
Stream mapping:
Stream #0:1#0:0 (aac (native) → pcm_f32le (native))
Press [q] to stop, [?] for help
Output #0, wav, to ‘/tmp/test.wav’:
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
ISFT : Lavf58.29.100
Stream #0:0(und): Audio: pcm_f32le ([3][0][0][0] / 0x0003), 44100 Hz, stereo, flt, 2822 kb/s (default)
Metadata:
handler_name : ISO Media file produced by Google Inc. Created on: 05/16/2019.
encoder : Lavc58.54.100 pcm_f32le
size= 15656kB time=00:00:45.44 bitrate=2822.4kbits/s speed= 620x
video:0kB audio:15656kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000711%

Yes, and it looks like things work correctly. So Ardour should be able to extract the .wav and import it (assuming you have disabled LTC analysis).

It is mysterious that this doesn’t work.

1 Like

@x42 “(assuming you have disabled LTC analysis)”

It worked! At least this once, I didn’t tick Extract LTC etc. and I got a sound track and sound!
I ran it from terminal, I’ll try it from apps, and with some other videos.

I’m sorry, I didn’t quite get your drift in earlier comments: DON’T EXTRACT LTC! Yes, that was the issue as I’d been doing that from the first. Like I said, I’m a noob… :confused:

@seablade I appreciate your efforts to help, sorry my second-guessing (extracting LTC every time) derailed all the efforts! I guess I’m good to go now. TX again.