AutoBatcher (batch file generator)

Simply put, "AutoBatcher" (open source) is a batch file generator for processing files in a folder. It was written as a direct response to my own limited knowledge of batch scripting which I find alien [e.g. for %%a in ("*.*") <do stuff>].

The most typical scenario I usually find myself in is that I would write a batch file that would do some kind of processing with ffmpeg to media files in a folder of certain file extensions (mkv, mp4, mp3, etc). This quickly gets out of hand when you find that you have a rapidly growing collection of batch scripts (not necessarily limited to processing with ffmpeg or media files).

And since my limited knowledge of batch scrpting forced me to concede that batch files can't do everything, I wrote "BatFunk" to complement AutoBatcher. With BatFunk, you can make batch files do stuff like perform mathematical operations, pause/sleep for x seconds, compute elapsed time, make a decision using 'iif' (like 'If-Then-Else'), and, oh yeah, probe a video for information (requires ffprobe).

BatFunk example to get the duration of a video:
%batfunk% ffprobe "durationhms/ ""%ifull%""/ "%ffprobe%"" > %tempfile%


AutoBatcher's notable presets:

Irfanview Slide Show
    - Launches Irfanview after creating a list of found images.

M3U Music Play List
    - Launches VLC after creating an M3U play list file of found audio files.

Search and TeraCopy Movies (with log file)
    - Uses TeraCopy to copy videos. The corresponding log file (file, size, start, end, elapsed) will be opened with Notepad after batch processing.

Make Movies LOUDER (AAC in MKV)
Make Movies LOUDER (MP3 in AVI)
    - Because non-audiophiles do not want to be bad neighbors. Get louder whispers and less jarring explosions through dynamic compression (personal recipe). Lets you watch movies late at night on LOW volume on your TV, tablet, or phone without missing important plot points.

VTT to SRT (SubRip subtitle)
Extract subtitles from MKV
MP4 + SRT into MKV
mp4toannexb
    - Other ffmpeg examples.

Enhance Dull Pictures
    - Uses ffmpeg's filters to enhance still images (personal recipe).

Video Encode Example
    - Uses ffprobe to get video info. Uses ffmpeg to resize the video. Sets window title to "x/y ~ duration ~ file" (duration on window title is a good substitute for a progress bar). The generated batch file also creates a corresponding log file (file, size, start, end, elapsed).

BatFunk's ffprobe functions and batchlog example (warning: slow)
BatFunk's ffprobe functions and batchlog example 2 (pipe-delimited batchlog)
    - Writes video information (requires ffprobe) into a log file.

Fix Portrait Videos (vertical videos)
    - Demonstrates the 'iif' function (like IF...THEN...ELSE statement). The generated batch file probes for video information (ffprobe), and then selects either Operation A or Operation B (ffmpeg). This is a personal recipe for processing tall videos (a cropped, wider, darker and blurred out version of the tall video will be used as background).

M3U Video Play List - Long Videos Only (duration greater than 1 hour)
M3U Video Play List - Short Videos Only (duration less than 1 hour)
    - Another demonstration of 'iif'. Batch file decides if video will be added to the play list.

* Irfanview, ffmpeg, ffprobe, ffplay, TeraCopy, and VLC are registered trademarks of their respective yada yada and all that...


Download AutoBatcherBatFunk:

AU3 Source Code Only Version:
https://drive.google.com/open?id=0BzyPuhJD-CzMamR2cjBpVm1Zckk

Executable (Windows):
https://drive.google.com/open?id=0BzyPuhJD-CzMUGxHWHNwSF9VQ00

Test Encode: Vandelay

Vandelay (fictional name) is a TV sitcom that ran for nine years.  Since my USB-equipped DVD player only plays xvid + mp3 in avi videos (burned as data dvd), the files (in mkv format) will have to be converted.  The episodes are organized by season, as shown below:

Burning just 5 episodes on DVD5 is wasteful (as with Season 1) and is not an option.  It is worth noting that not all episodes are of equal duration and that some files are actually two episodes merged into one file.  In this case, Vandelay has a total of 180 episodes in 171 files.

Clicking "Select folder" ('step 2' of En Masse's main form) reveals that the Vandelay folder contains 171 files with a total duration of 67h 17m 19s.  Clicking 'Duration/x' reveals that opting for 6 discs would yield a video bitrate of 765 kbits/s, an audio bitrate of 128 kbits/s, and a maximum duration of 11h 25m 25s on DVD5.

After deciding to go with 6 discs, I then set out to re-organize the Vandelay folder.  I created six folders, and in conjunction with 'GetTotalDuration.exe', placed the maximum amount of files in each folder that would yield a total duration not exceeding 11h 25m 25s.

As shown above, episodes 1-31 (Disc 1) yields a total duration of 11h 20m 30s.

I then created a new preset with bitrate settings as suggested by 'Duration/x' (765 + 128 kbits/s), and went through steps 1-4 (select preset, select folder, generate, convert).

After ffmpeg processed the files for 40h 30m 24s (real files were used, btw), I ended up with the following:

Download VidProkEnMasse:
http://vidprokenmasse.blogspot.com/2014/07/vidprokenmasse.html

VidProk: Preview ffmpeg filters

VidProk lets you preview ffmpeg filters (no more test encodes, yey!).  Additionally, it includes a VF cheat sheet which lets you conveniently copy/paste filters.

Some of the video filters include:  Add noise, color curves, deinterlace, denoise (hqdn3d), deshake, draw box, draw grid, fade in, flip horizontal, flip vertical, gamma, geq - emboss, grayscale, histogram, hue and saturation, mplayer: gamma, brightness, contrast (experimental), negate, pp - deblocking, deringing, auto brightness & contrast, sepia, smartblur, unsharp, vignette, and YUV420P.

The VF cheat sheet is by no means a substitute for ffmpeg's documentation --- so, if the default values do not satisfy you, read the ffmpeg manual.

Here are some screen shots...

'play both (horizontal)':

filters used: gamma, curves, saturation, unsharp, denoise, deblock, yuv420p

'play both (vertical)':

And finally, 'play both @ pos 0x0':  Both source and processed windows are placed on the top left corner (one window obscures the other).  Clicking 'toggle' alternates between the source and processed windows.  This lets you easily see the differences, especially small differences, between the processed and the source windows.

 I know that ffmpeg is a no-joy in the instant gratification department, but I hope you'll like using this tool.

Youtube demo:
http://youtu.be/zgXiGfZlTVo

VidProk is included in 'VidProkEnMasse', it is absolutely free, and you can download it here:
http://vidprokenmasse.blogspot.com/2014/07/vidprokenmasse.html

VstatsReader

VstatsReader is a simple CLI app which reads ffmpeg-generated video statistics logs (vstats_*.log) and retrieves information like peak q, peak bitrate, etc.  It generates an ini file which will be located in the same folder as the vstats log file (same file name with an 'ini' extension).

Here's a sample of a generated ini file:

[Vstats] VstatsLogFile=D:\SEKI APPS\Work-In-Progress\VstatsReader\sample.log LowQ=2 AverageQ=6.04 PeakQ=7.9 LowBitrate=8889 AverageBitrate=11279.3 PeakBitrate=14973.1 ApproximatePositionOfPeakBitrateAlongTimeline=0.01 Frames=304 VideoStreamSizeMB=13.63 start=2014/07/10 19:50:59 end=2014/07/10 19:51:13 elapsed=0h 0m 14s ; bitrates are in 'kbits/s'.

Q stands for quantizer.  "ApproximatePositionOfPeakBitrateAlongTimeline" (yeah, I know it's a mouthful) shows a number which you can use to compute when or where along the timeline of the video the peak bitrate occurred.  If this value is 0.5 and video duration is 2 hours, then the peak occurred at 3600 seconds or at 1 hour, 0 minutes and 0 seconds.  This is, of course, just an approximate.  The reason why I elected to use a number (instead of time/PTS) is that vstats logs seem to show erroneous data when "-f rawvideo NUL" was used (even average bitrate seems to be off).

Usage:
1.  In Windows Explorer, you can drag & drop a vstats log file onto the file "VstatsReader.exe".
2.  In a batch file:

@echo off set vstatsreader="C:\VstatsReader\VstatsReader.exe" set vstatslogfile="E:\Video Processing\Output\Logs\Debbie went to Dallas (2014).log" %vstatsreader% %vstatslogfile%

Here's an example of an ffmpeg command that will generate a vstats log file:

%ffmpeg% -i "E:\MOVIES\Tiktik - The Aswang Chronicles (2012) [Tagalog].mp4" -vf "scale=640:360, setsar=1, format=pix_fmts=yuv420p" -vtag xvid -g 300 -qscale:v 4 -vstats -vstats_file "E:\~ VidProkEnMasse\OUTPUT\Logs\Tiktik - The Aswang Chronicles (2012) [Tagalog]_FMP4_vstats.log" -c:v mpeg4 -r 29.97 -c:a libmp3lame -b:a 96k -ar 48000 -ac 2 "E:\~ VidProkEnMasse\OUTPUT\Tiktik - The Aswang Chronicles (2012) [Tagalog]_FMP4.avi"

Other files included:
'sample.log' - you can drop this file onto the file 'VstatsReader.exe' to test.
'VstatsReader.au3' - source code (AutoIt).

Download:
http://www.mediafire.com/download/n21tw3jer1ta2mn/VstatsReader.zip 

Batch denoising JPEG files with ffmpeg

I love PhotoScape! If your photoshop skills are not up to par, then you will like this program too. Its 'film effect' filters are amazing, but what I really like about this program is its Noise Reduction (clear skin) feature - your woman will love you for using it, especially on close-ups (less visible blemishes/pores/lines). However, this functionality is not available in batch editor mode (last time I checked). *sigh*

Fortunately, ffmpeg offers the next best thing, and it's called High Quality 3D Denoise.

Just create an empty text file, rename it to, like, "Denoise_JPEG_files_in_this_folder.bat", copy/paste the code below, set the appropriate ffmpeg path, tweak to taste, copy/paste the batch file into a folder full of jpeg files, and you're ready to go!

BTW, set the value for "AdditionalVideoFilters" to blank if you don't want the set filter.

<code starts after this line>

@echo off

: _______________________________________________

: This batch file will apply ffmpeg's High Quality 3D Denoise filter on all found image files in the current folder (subfolders included) and open the output folder. Don't forget to set ffmpeg's full path.

: _______________________________________________


: enter required/desired settings

: set ffmpeg path (required). use double quotes.
set ffmpeg="C:\~ En Masse\third party bin\ffmpeg-20131222-git-9b195dd-win32-static\bin\ffmpeg.exe"

: set LumaSpatial. practical range for a 640x480 image is 4-12. no quotes.
set LumaSpatial=12

: set image file types to search/process. note that all files will be converted to the set outputfile extension. no quotes.
set FileTypes=*.jpg, *.jpeg, *.bmp, *.png, *.tif, *.tiff

: output folder (subfolder). no quotes/spaces.
set OutputFolder=~Denoised

: output file name suffix. no quotes.
set Suffix=_Denoised

: output file extension (jpg, png, bmp, etc). no quotes.
set OutputExtension=jpg

: additional video filter (optional). this is for demonstration purposes only. precede each filter with a comma ','. you can set the value to blank if you don't want this feature. no double quotes at both ends.
set AdditionalVideoFilters=, curves=master='0.05/0.045 0.06/0.06 0.5/0.6'

: _______________________________________________

title %~n0

: create the output folder
md %OutputFolder%

: open the created output folder
start explorer.exe %OutputFolder%

: search and process
FOR /F "delims=" %%A IN ('dir /a:-d /b /o:n /s %FileTypes%') DO (
echo %%A
%ffmpeg% -threads 1 -y -f image2 -i "%%A" -f image2 -qscale:v 1 -vf "hqdn3d=luma_spatial=%LumaSpatial%: chroma_spatial=1: luma_tmp=1: chroma_tmp=1 %AdditionalVideoFilters%" "%OutputFolder%\%%~nA%Suffix%.%OutputExtension%"
)

exit

A desperate method to burn-in subtitles

While the method may seem 'desperate', this may also prove to be the easiest way to burn-in subtitles (*.srt).  It seems to me that any thread on any forum about burning in subs with ffmpeg appears to be silent.  The lack of eureka's and heartfelt thank-you's only confirms one's suspicion that the thread topic is unresolved.  In weary resignation, I have concluded that: "It's just me... and everyone else can do it."

Reading up on the somewhat cryptic ffmpeg documentation, I learned that the drawtext filter may probably be only good for PTS, timecode, and single-line text.  The proverbial lightbulb somehow flickered when I came upon the sendcmd filter.  Hmmm...  I imagined hearing the sound of rusty gears slowly turning in my head.  The quietest most hesitant 'eureka' in the history of mankind may have escaped my lips.  Uhh no, I didn't actually say 'eureka', because I'm Filipino, and we don't talk like that, but you get my drift.

After some quick tests, the verdict was out:  All one has to do is create a sendcmd file (*.txt) containing all the drawtext commands telling ffmpeg what text to draw and when to draw it.  It's quite simple really!

The idea is to change this:
1
00:00:19,420 --> 00:00:23,090
A SERBIAN FILM

into this:
00:00:19.420-00:00:23.090 [enter] drawtext reinit text='A SERBIAN FILM', [leave] drawtext reinit text='';

With less than 300 lines of code (AutoIt), I have managed to convert an *.srt file into a text file that could boss around ffmpeg.

This is my eureka moment I want to share with anyone who has lost a good amount of hair trying to figure out how to burn-in those d*mn subtitles.

This tiny app (absolutely free) for Windows is about 641 kb (includes the CLI executable, a GUI, and source code) and you can download it here:

updated: 2014.05.26
http://www.mediafire.com/download/3is5jc5v851h274/srtdrawtext_2014.05.26.zip

An easier-to-use version that works in conjunction with VidProk (included in 'VidProkEnMasse' which is also absolutely free) can be downloaded here:  http://vidprokenmasse.blogspot.com/

Update 2014.05.26:
Yeah, I made a boo-boo... I didn't bother googling SubRip specifications prior to coding with a handful of SubRip files as samples (hehehe oops!).

VidProkEnMasse

VidProkEnMasse is an all-ffmpeg solution for your daily video encoding needs.  The name is derived from its two major apps in the collection.  "VidProk" processes videos 'piece-meal' while "En Masse" batch processes videos (last tested on 1849 files).

Apart from their intended use, VidProk and En Masse are also excellent learning tools for new ffmpeg users as both apps readily reveal their ffmpeg 'recipes'.  While the GUI provides convenience for experienced ffmpeg users, you don't have to be an ffmpeg guru to use VidProkEnMasse.  It does not include a large collection of device-specific presets, but it fairly outlines some of ffmpeg's vast capabilities.  You can consider the example presets as a sort of walk-through.

VidProkEnMasse is a home-brew solution created by a frustrated end-user for other end-users.  It does not write to the registry, does not automatically 'phone home' for updates, does not nag, does not coerce your browser to a new home page/search engine or to install a new toolbar, and does not require you to join a cult.  VidProkEnMasse is absolutely free, though small donations are very much welcome.  Email me: chiqboy<[at]>gmail<[dot]>com

En Masse features:
1.  Easy operation:  Select preset, select folder, generate batch file, convert.
2.  The usual features:  Burn-in *.srt (SubRip) subtitle files, single-/two-pass encoding, crf-/quantizer-/bitrate-based encoding, generate video statistics logs, shutdown after encoding, etc.
3.  Logged encodes:  Elapsed time, render speed, rendered file size, possible failed encode, etc.
4.  Preset Manager - Bitrate-based encodes: Integrated qf calculator.  Max duration on DVD-5/-9.  Resize all videos to one width or depending on aspect ratio, or no resize at all (square pixel/anamorphic).
5.  Queue Stats readily displays total number of files/size, total duration, and estimated total output size (if available).
6.  Integrated bitrate calculator (Duration/x).  Suggests video bitrate based on total video duration (as on Queue Stats) and the number of discs you specify.
7.  Text expansion support (%variables%).

VidProk features:
1.  Easy operation:  Drag/drop video, select preset, add video filters, tweak, preview, encode.
2.  The usual features:  Burn-in *.srt (SubRip) subtitle files, single-/two-pass encoding, crf-/quantizer-/bitrate-based encoding, extract frames to jpg files, extract screen shots, extract embedded subtitles to srt, extract audio to wav, change container or discard/copy stream, generate video statistics logs, crop/scale/distort frames, select/tweak/preview video filters, etc.
3.  Logged encodes:  Elapsed time, render speed, rendered file size, possible failed encode, etc.
4.  Text expansion support (%variables%).
5.  Unique feature:  LAP (low, average, peak).  Answers the question: 'What's the peak video birate like if I were to encode with XYZ settings?'.

Some of the other tools included:

GetTotalDuration.exe:  Computes total and average duration of dragged/dropped video files.

JoinAVstreams.exe:  Stream copy/mux.  May fix generally faulty videos or some rogue mkv files (variable frame rate with missing frame count).

AnalyzeVstatsLogFile.exe:  Similar to "LAP".  Reads ffmpeg-generated video statistics logs and retrieves low/average/peak q, low/average/peak bitrate, etc.  Ever wondered what the bitrate is like for T.V. noise?


En Masse demo:
https://www.youtube.com/watch?v=6pSH8lQZpRQ

Download VidProkEnMasse:

Click Here

Update: VidProkEnMasse_20170116
1. Modified scale routine: If Width combo boxes are left blank, no scale filter will be applied. I was compelled to implement this when I processed videos from a smart phone (annoying "portrait" videos).

2. Added "SRTFix.exe" (GUI) and "SRTFixCLI.exe" (command line). Both attempt to fix subtitles (*.srt) with long lines that go right off the screen. Limits subtitles to two rows when max characters is exceeded.

3. Added "GaugeVideoGenerator.exe". Default settings attempt to replicate the visual content (bitrate-wise) of "Transformers - Age of Extinction (2014)"... shortened to 100 seconds. Lets the user guesstimate output file size of crf-/quantizer-based encodes. Said movie, AFAIK, has the highest bitrate for a feature film. May answer the silly question: "What is the equivalent bitrate of crf 23?"

4. Added %presetdetails% to text expansion. Useful when used on metadata comment.

%presetdetails% expands to:
PresetName: V/A codec: vc+ac. V/A bitrate: vb+ab. fps: x. Widths: a, b, c, d. PIFA: xyz. VFA1: xyz. VFA2: xyz. PvfFFA: xyz.

5. Added %loud% to text expansion which makes audio output significantly louder

%loud% expands to:
-af "aformat=channel_layouts=stereo, compand=0 0:1 1:-72/-72 -48/-6 -3/-3 24/-3:0.01:0:-96:0"

6. Updated to "ffmpeg-20160116-git-d7c75a5-win32-static" which works on my OS (Windows XP 32-bit). Please feel free to replace with the latest ffmpeg Zeranoe static build if desired/required.

7. Unfortunately, VidProkEnMasse, being written in AutoIt, still suffers false positives from lazy AV software companies :\


Update:  VidProkEnMasse_20140714
1. Renamed app to 'VidProkEnMasse'.
2. Added text expansion feature (%variables%).
3. Added 'AnalyzeVstatsLogFile.exe' which retrieves info (like peak q, peak bitrate, etc) from vstats log files.
4. VidProk: Added 'LAP' (Low, Average, Peak) which performs video statistics gathering and retrieves info similar to 'AnalyzeVstatsLogFile.exe'.
5. Added 'Longest.Line.In.Srt_[DROP.SRT.FILE.HERE].exe'.
6. Numerous tweaks.
7. Updated to ffmpeg-20140712-git-ca671be-win32-static.
8. Moved site to http://vidprokenmasse.blogspot.com/2014/07/vidprokenmasse.html

Update:  EnMasse_20140528
1. Dropped MediaInfo in favor of ffprobe (En Masse is now an ALL-ffmpeg solution).
2. En Masse: Burn-in subtitles (batch mode).
3. New policy: Maintain anamorphic dimensions if not resized.
4. Two-pass encoding.
5. VidProk encoding is now logged.
6. Numerous tweaks.
7. Updated to ffmpeg-20140502-git-0f9f24c-win32-static.