Stable video indexing in Fast video cataloger 8

We have worked very long to make sure Fast video cataloger indexing is stable and can handle just about all video formats.

Indexing videos are always going to be a pretty time-consuming process since the video file needs to be read and decoded. Just reading through the large files is a time and CPU-consuming process. Having videos on a fast device and having a fast multicore CPU will help but even so, if you have many videos it will take considerable time.

From the start, we made sure you can add a list of files to index so that you can let it work along if you have hundreds or thousands of videos to index. After each video is indexed the current playlist is saved so the program can continue where it left off in case the program crashed, the power went out, Windows decided it was the perfect time to do an update, or the program closed down for any other reason.

We have put in plenty of effort to fix bugs in our software to make this as stable as possible. We also code to handle broken videos, conflicting codecs, and so on. But even so, we have reports of people getting crashes when they index large lists of videos.

So what happens when a video is indexed and what is the problem here?

What happens during video indexing?

To decode a video we first need to load and parse the video file format. A video file, i.e the container, contains video, audio, and perhaps other data. So the first thing we need is to read the data in the container and split it out into its different parts. How the format stores its data is dependent on the video format i.e. mp4, MPEG, Avi, etc. This piece of container parsing code is also sometimes called a splitter as it is splitting up the video file into separate streams of the same data type i.e streams for audio, video, etc. Then we need a video decoder for the video stream. This basically takes the video stream and reads it in compressed format and decompresses it. There are a whole lot of different compressions out there and they are constantly being improved upon. Typical video compression is lossy and uses similar techniques as image compression, like jpeg for each frame in the image and also the fact that not every single pixel changes between frames. Some of the compression algorithms have been implemented in hardware on GPU, so for even speedier decompression the decoder can in some cases use hardware decompression, in that case, you also have a video driver involved and the driver depends on the type of video card you have installed.

After the decompression, we have an uncompressed image of a video frame. This is basically all we need for the indexing process, we compress the image again and it is saved in the video catalog.

What can go wrong?

So to summarize, every time a video is indexed, in addition to Fast video cataloger there is a splitter for the format of the video, a decoder for the compression, and possibly a video driver. How these different components run also depends on parameters given to the container format and compression. So for every video, there is a risk that there is an issue in the splitter, decoder, or driver and the bug might not be an obvious crash. It could be a random memory overwrite that messes it up for the following video or another of the components.

The more videos you index in a row the higher is the risk that the computer encounters something unexpected that goes wrong.

And

There is basically no way we can prevent this from happening.

The Fast video cataloger 8 solution

What we can do is to isolate the whole indexing process from the rest of Fast video cataloger and each indexed video from the next. We basically launch each indexing in a separate process so that we always start from a known state and so that each indexing of a video is separate from the other. As a user you should not see any difference, speed is the same as before and you can follow the progress as the video is indexed.

This has now been implemented in Fast video cataloger 8 that you can download from here: https://videocataloger.com

video indexer settings

Settings for indexing videos in fast video cataloger

Fast Video Cataloger 8 available for download

Fast Video cataloger 8 is a local video content management system. Have instant access to all your videos from one single place and produce content quicker than ever before.

We started development in 2011 and today we are happy to finally release version 8. The program is available now for Windows PC from https://videocataloger.com/download/.

Version 8 is a major update and comes with more than 100 improvements. Enhancements to the playlist feature help editors to quickly draft up outline suggestions for exports into editing software like DaVinci Resolve.

Automatic video metadata extraction now supports XMP and Windows metadata as well as further customization of extended properties.

We have written a summary of everything that is new in version 8 that you can view here.

The documentation explains the program in detail, you can download it separately or download the full software package including the documentation.

Video Cataloger 8

fast video cataloger 8 is now available for download

Using web browser from scripts

Using a web browser in Fast video cataloger

The start/help window in Fast video cataloger is a web browser. It uses the chromium embedded framework and you can access and use it easily from scripts.

You get the browser interface from the standard scripting interface

var browser = scripting.GetBrowser();

This is a standard chromium (chrome) browser interface and the official documentation can be found here:

To open a web site in the browser you would simply write:
browser.Address = "https://google.com/";

And here is a full example:

using System.Runtime;
using System.Collections.Generic;
using VideoCataloger;
using VideoCataloger.RemoteCatalogService;
class Script
{
  static public async System.Threading.Tasks.Task Run ( IScripting scripting, string arguments ) 
  { 
    var browser = scripting.GetBrowser();
    browser.Address = "https://google.com/";
  }
}

The use of the browser has almost endless possibilities. You can use it to present information and info from your scripts, you can fetch data from other systems or even create custom user interfaces.

Search Video Actors

Search video actors with Google

This text will show you step by step how to add a custom button in Fast Video cataloger that will let you google results for the currently selected actor.

Custom Actions in Fast video cataloger allow you to easily extend the functionality.

You can extend the program to launch web pages, launch external programs, or for more advanced scenarios launch c# scripts with full access to the complete video database.

In this example, I will show how you can search google on the currently selected Actor.

The basic Googles URL for searches is this:

https://www.google.com/search?q=

The text after q= is the query, and if you have more than one term just use a + in between them.

In Fast video cataloger, go to preferences and to the Action Tab.

Action tab in preferences

Click “Add” to create a new action.

On the first tab, Enter “Google Actor” as a label. And for tooltip enter “Google the selected Actor”

Title for the actor search button

Click on the “Execute” tab and change the drop-down list to “Web Page”

For URL Enter: https://www.google.com/search

site of the the search engine

Click the Arguments tab.

For arguments enter : q={actor_firstname}+{actor_lastname}

Enter the url arguments to pass in the first and last name of the actor.

Click OK to close the window. And ok again to close the preferences.

From the view menu, open the Action window.

Open the “Custom Actions” expander and you should have a button “Google Actor”

The new search actor button is in the action window.

Select an Actor in the Actor window, click the “Google Actor” button and the browser window in fast video cataloger should show you the result.

Fast video cataloger with Microsoft intune

What is intune?

Intune is a relatively new application deployment technology from Microsoft.

“Microsoft Intune is a cloud-based service that focuses on mobile device management (MDM) and mobile application management (MAM). You control how your organization’s devices are used, including mobile phones, tablets, and laptops. You can also configure specific policies to control applications.”

Can Fast video cataloger run with intune?

You can package the Fast video cataloger installer as a .intunewin file using the Microsoft Intune Win32 App Packaging Tool.

Microsoft Intune Win32 App Packaging Tool will check the Windows Classic setup files and generate a .intunewin file imported into Intune Azure Portal.

You can download the tools from from Microsoft here:  https://github.com/Microsoft/Microsoft-Win32-Content-Prep-Tool

You will need to agree to their license agreement.

Read more about the tool here: https://docs.microsoft.com/en-us/mem/intune/apps/apps-win32-prepare

You can download the Fast video cataloger installer from the Fast video cataloger download page.

You run their tool from the command line and it is pretty straight forward.

Silent or unattended installation of Fast video cataloger

Fast video cataloger uses nsis as an installation package. You can download the latest Fast Video Cataloger installer from our download page. We always sign our installer and run it through antivirus software.

If you want to run the installer silently simply start it with /S (upper S) as a command-line argument.

If you want to run the installer so it installs into another output folder give it the /D argument. /D=<installation directory> to change the default installation directory.

Add keywords from video filename

Have you ever wanted to create keywords based on the filename of a video?

With the scripting support in Fast video cataloger, this is actually pretty easy. Here is a sample script to do just that, create keywords from the filename or path of a video.

The provided script takes the currently selected videos and adds keywords and actors based on the name of the video file.

You might want to tweak the script a bit to suit your own naming standard.

The array of separators list the characters that should be treated as keyword separators.

The array ignore_words is a list of common words that should simply be ignored and not added as keywords.

The integer min_length is the number of characters that a word needs to contains to be added as a keyword.

Finally, if a string contains the @ character that sentence will be treated as an actor. First name before the @ and last name after the @. The actors are created and added as cast to the video.

As usual, to run a script in Fast video cataloger you load it into the console window and click the Run button.

Script console and web browser open.

 

#region title_to_keywords


using System;
using System.IO;
using System.Runtime;
using System.Linq;
using System.Collections.Generic;
using VideoCataloger;

/// <summary>
///  Take the title and use it to generate keywords.  
///  If there is a @ in the title we treat that as an actor
///  the text before @ is first name and the text after is last name
///  if an actor with that name already exist we use that one.
/// </summary>
class KeywordsFromTitle
{
    static public void Run(IScripting scripting, string arguments)
    { 
        scripting.GetConsole().Clear();
        var catalog = scripting.GetVideoCatalogService();
        ISelection selection = scripting.GetSelection();
        List<long> selected = selection.GetSelectedVideos();
        foreach (long video in selected)
        {
            // Get the video file entry
            var entry = catalog.GetVideoFileEntry(video);
            scripting.GetConsole().WriteLine(System.Convert.ToString("Processing..." + entry.FilePath));

            char[] separators = { ' ', ',', '.', '-', '[' ,']', '{', '}', '_' };
            string[] ignore_words = { "is", "are", "who", "where" };
            string title = entry.Title;
            string[] keywords = title.Split(separators);
            int min_length = 3;
            foreach (string word in keywords)
            {
                if (word.Length>= min_length)
                {
                    if (!ignore_words.Contains(word))
                    {
                        if (word.Contains("@"))
                        {
                            // Actor
                            string[] names = word.Split('@');
                            string first_name = names[0];
                            string last_name = names[1];

                            scripting.GetConsole().WriteLine( "Actor FirstName:"+ first_name + " LastName:" + last_name );

                            int actor_id = -1;
                            VideoCataloger.RemoteCatalogService.Actor[] current_actors = catalog.GetActors(null, first_name, last_name, true);
                            if (current_actors.Length >= 1)
                            {
                                actor_id = current_actors[0].ID;
                            }
                            else
                            {
                                VideoCataloger.RemoteCatalogService.Actor actor = new VideoCataloger.RemoteCatalogService.Actor();
                                actor.FirstName = first_name;
                                actor.LastName = last_name;
                                actor_id = catalog.AddActorToDB(actor);
                            }

                            if (actor_id != -1)
                                catalog.AddActorToVideo(video, actor_id);
                        }
                        else
                        {
                            // Keywords
                            scripting.GetConsole().WriteLine("Keyword:" + word );
                            scripting.GetVideoCatalogService().TagVideo(video, word);
                        }
                    }
                }
            }
        }

        // refresh the gui to show the changed file paths.
        scripting.GetGUI().Refresh("");
    } 
}
#endregion

Search for video file name

How to search for a video file in a catalog

Here is a short sample script to search for a video file in the catalog given a filename.

Simply use the search api and provide the VideoQueryProperties. FilePath is a part of the video path you are searching for.

If the search failes to find any videos it will return null.

If it finds videos it will return an array of VideoFileEntry. The entries includes, among other things, the ID of the video. You use the id as key for the other APIs.

As usual, copy paste the script into the script window in Fast video cataloger. This is just a small sample, if you are developing your own script make sure to check our other developer resources.

using System.Runtime;
using System.Collections.Generic;
using VideoCataloger;
using VideoCataloger.RemoteCatalogService;
class Script
{
  static public async System.Threading.Tasks.Task Run ( IScripting scripting, string arguments ) 
  { 
    scripting.GetConsole().Clear();

    VideoQuery query = new VideoQuery();
    query.Properties = new VideoQueryProperties();
    query.Properties.Description = "";
    query.Properties.Link="";
    query.Properties.Title = "";
    query.Properties.FilePath = "part_of_filename";
    VideoFileEntry[] result = scripting.GetVideoCatalogService().SearchVideos( query );
    if (result!=null)
    {
      foreach (VideoFileEntry entry in result)
      {
        scripting.GetConsole().WriteLine( "ID=" + entry.ID );
      }
    }
  }
}