A few Linux utilities that are useful for manipulating XMLTV schedule files

In my previous article, Some hints for getting free-to-air satellite channels into the Electronic Program Guide in Kodi or XBMC (or another frontend), I mentioned that schedule “grabber” programs save their files in XMLTV File format. So let’s say you have one or more XMLTV type files, but you want to do some additional manipulation on them before feeding them to your backend software.  Here are a few tools that run under Linux that I have found that may be useful, under certain circumstances.  These are in addition to zap2xml, which I mentioned in my previous article.

NOTE:  To find out if you have a particular program installed on your system, try entering the word which followed by a space and the program name at a Linux command prompt.  If the program is installed on your system, it should show you the path to the file.  Note that you will probably need to use the full path and filename if you are attempting to run the program from a shell script or a cron job!

  • tv_cat – Concatenate XMLTV listings files. The man page description says, “Read one or more XMLTV files and write a file to standard ouput whose programmes are the concatenation of the programmes in the input files, and whose channels are the union of the channels in the input files.”  Or in simple terms, it merges XMLTV format files together.  This program may already be on your backend system but if it’s not, you can typically install it on a Ubuntu/Debian-based system (and possibly in some other Linux distros) by installing the xmltv package.  tv_cat is a bit picky about the format of the files it will combine, so check the output carefully to make sure it is including all the channels.  I had some issues using tv_cat with TVHeadEnd, and wound up using a small quick-and-dirty shell script to combine XMLTV listings files instead (see below).
  • sed – Stream EDitor. This is a utility built into just about EVERY Unix/Linux system out there, and it’s probably available for Windows in some form also. sed is more or less a one-trick pony – it searches for text and replaces it with something else.  You can use it to resolve duplicate channel ID’s in two different XMLTV files by changing them in one of the files, using a command of the form sed -i ‘s/original text/replacement text/’ filename but note that there are some potential “gotchas”, so read the documentation first.  For example, if either the search or replace string contains a / character, it mush be “escaped” with a backward slash, so as not to be confused with the / delimiter character.  So if, for example, your search or replace string included the closing tag you’d use instead. (EDIT: You can also change the delimiter character to avoid this issue – see the first comment below).

    I will note that there are some “purists” out there that will say that you should never use sed to manipulate an XML file, even though it’s easy and (if you are careful to use unique strings that don’t appear anyplace that you don’t want to change) fairly foolproof, so I suppose I had better mention a tool that is specifically intended for manipulating XML files…

  • xmlstarlet – command line XML toolkit. According to the description, “XMLStarlet is a set of command line utilities (tools) which can be used to transform, query, validate, and edit XML documents and files using simple set of shell commands in similar way it is done for plain text files using UNIX grep, sed, awk, diff, patch, join, etc commands.” This is another one that you will likely find in your Linux distribution’s repository, at least if you are running a version of Ubuntu or Debian.  This one offers you a lot more flexibility in manipulating XML files, but at the expense of being somewhat more complicated to use.

Since xmlstarlet is a bit difficult for some users to wrap their heads around, I will give here some actual examples of how it could be used on an XMLTV format file, but please note that I am no expert with this so if you have a different proposed usage, please try to figure it out for yourself using the handy documentation, available as a web page or in PDF format.  No offense, but better you should spend a couple hours trying to figure out the correct syntax to achieve whatever results you want than me! 🙂

1. Remove all “Local Programming” entries from an XMLTV file named xmltv.xml and save to newfile.xml:
xmlstarlet ed --delete "//programme[title='Local Programming']" xmltv.xml >newfile.xml
2. Same as above but only for one specific channel:
xmlstarlet ed --delete "//programme[title='Local Programming'][@channel='someid.someaddress.com']" xmltv.xml >newfile.xml
3. To change the value of @channel wherever it appears in the file:
xmlstarlet ed -u "//programme[@channel='someid.someaddress.com']/@channel" -v 'newid.newaddress.com' xmltv.xml
4. To extract all entries for a specific channel to a separate file (non-destructive – does not change the original file):
xmlstarlet sel -t -m "//programme[@channel='someid.someaddress.com']" -c . -n oldfile.xml >newfile.xml

Note that I am not saying that any of the above are the best example of how to do something.  As you can see, especially from the last example given, this program has some rather non-intuitive syntax for its command line arguments (to put it mildly).  If you have any additional – or better – examples of using xmlstarlet to manipulate XMLTV files, please leave them in a comment and I will consider adding them here.

That said, if you need to do an operation on an XMLTV file and don’t want to write a program or script to do it yourself, xmlstarlet could be your salvation – IF you can figure out how to use it!

Now, assuming you are not one of the “purists” that recoils in horror to the idea of editing an .xml file as if it were a text file, here is how you can use a Linux shell script and sed to combine two or more XMLTV format files. This assumes that each file has a header of exactly four lines, the last of which (the fourth line) starts with “<tv", and that each file ends with "” as the final line. It also assumes that you want the output file to be named tv_grab_file.xmltv (if you use Tvheadend, this may be the name you want).

cd /path/to/xmlfiles
sed '$d' first_file.xml > tv_grab_file.xmltv
sed --separate '1,4d; $d' second_file.xml third_file.xml last_file.xml >> tv_grab_file.xmltv
echo '' >> tv_grab_file.xmltv

In the second sed line above, you can list from one to as many additional files as you want to combine with the first file, but if there is more than one file listed on that line then you must use the “–separate” option to remove the headers and footers from all files. And yes, I do realize that instead of using an echo command to write the final line, I could put the final file in that line, like this …

sed '1,4d' last_file.xml >> tv_grab_file.xmltv

… and it would preserve the “” line from that file, but the reason I don’t do it that way is because if for some reason that final file is missing, the “” line would not get written and that xml block would not be closed, which might cause some processors to ignore it completely. I do realize that the same problem exists if the first file happens to be missing, but in my installation that is far less likely to happen, for reasons I won’t go into here. But if you are concerned about it, you could write all four of the header lines using echo statements, then put all your .xml files in a single sed command (like the second one in the above script).


Some hints for getting free-to-air satellite channels into the Electronic Program Guide in Kodi (or another frontend)

If you are running a satellite backend system such as Tvheadend or MediaPortal (or MythTV, if you are one of the lucky few that can actually get it to work), and you use Kodi or the MythTV frontend, then it is possible to populate the schedule grid with listings from many sources. Note I did not say that it is easy, just that it is possible. The key is to use an external program such as zap2xml (Zap2it TV listings to XMLTV or XTVD .xml). These are commonly referred to as “schedule grabbers”, or just “grabber” programs.

The real trick is figuring out how to use one of those programs. Typically they are used to grab listings for a single over-the-air market, not a hodgepodge of stations and services from various locations. Such programs will create a xml file that contains schedule listings (in Tvheadend it will typically be at /home/hts/.xmltv/tv_grab_file.xmltv, assuming that “hts” is the Tvheadend username on your system), and that file will have all the over-the-air channels in your area, or all the cable or satellite channels from your provider. If you want to use zap2xml and you’ve never set it up before, I’ll give you some setup hints later in this article. But for now, lets assume that you have it all set up and you know how to create an xml file (named /home/hts/.xmltv/tv_grab_file.xmltv) containing your local listings.

Once you have created the xml file, it can be imported into the Tvheadend, MediaPortal, or MythTV database. There are now two different methods for importing channels into Tvheadend, a newer method that uses a socket, which I will discuss momentarily, and an older method that uses a file called tv-grab-file, which must be downloaded and moved into the /usr/bin directory on the system running Tvheadend (be sure to make it executable, since it is a bash script). Also, you may need to edit the line in the script that starts with “cat” and contains the path to the tv_grab_file.xmltv file, to specify the correct path and file name of the file produced by your selected grabber program, if it isn’t being saved as /home/hts/.xmltv/tv_grab_file.xmltv. Note that if the tv_grab_file.xmltv file is not owned by the Tvheadend user, it must at least be made readable by Tvheadend – incorrect permissions and/or ownership on this file will make it inaccessible to Tvheadend.

Note that in the newest versions of Tvheadend, the grabber settings have been broken up into three tabs, labelled “EPG Grabber Channels”, “EPG Grabber”, and “EPG Grabber Modules”, but you may not be be able to see all of the grabber settings until you do this: Click on the Configuration tab, then the General tab, then the Base tab, and then in the “User interface level” setting choose “Expert” from the dropdown. Then click “Save” (directly under the “Base” tab). Now you should be able to see all the EPG Grabber settings. In this new interface, you select the grabber in the “EPG Grabber Modules” tab, and all of the grabbers listed there should be disabled (orange dot at start of line) except for either “External XMLTV” (if you plan to use the socket method) or “Internal XMLTV: tv_grab_file is a simple grabber …”. Whichever you select should have a green dot at the start of the line (click on any dots that are the wrong color to change them, then don’t forget to click the “Save” button). This screenshot shows tv_grab_fle selected:

Once you have done that, you tell Tvheadend to use the tv_grab_file script by going to Configuration | Channel / EPG | EPG Grabber page. First, if any grabbers are enabled in the “Over-the-air Grabbers” section, disable them by unchecking the boxes next to them. Then, in the “Internal Grabber” section, select “XMLTV: tv_grab_file …” in the dropdown (you may need to restart Tvheadend or reboot the system before it will appear – if it still doesn’t appear, check the ownership and permissions of the file, it should be the same as the other tv_grab_* files in the /usr/bin directory – owned by root, and executable by all users). This is how it looked in previous versions of Tvheadend:
Tvheadend Internal Grabber SettingsIn the newest versions of Tvheadend you can actually schedule the Internal Grabber to import the listings at a specific time each day. Here I have set it to run at 2:33 AM every day (after commenting out the default):

NEW Tvheadend Internal Grabber settings

You set the time you want the grabber to run in the “Over-the-air grabbers” section of the “EPG Grabber” tab. I will spare you my usual rant about developers making changes in the user interface that make things more confusing for users, but suffice it to say that it appears that every new version of Tvheadend changes the user interface in significant ways, so you may have to look around a bit to find these settings.

If you would prefer to try the newer method that uses a socket, then in the “EPG Grabber Modules” tab you would disable any other grabbers and enable only “External XMLTV” by checking the “enabled” box in the screen shown below (then, don’t forget to click the “Save” button). But make a note of the path shown in the “Path” field; you will need to know that later.

Now run the following commands from a Linux command prompt – the responses you get will determine which method you use to import the guide data into Tvheadend:

which socat
which nc
which curl

Ignore the ones that do not return a path to the program mentioned. For example, typically if you run “which nc” you will get a response such as /bin/nc – that indicates that the “nc” (netcat) program is installed on your system.

If typing “which socat” returned a path, then you can use this to import your guide data into Tvheadend using the socket method, replacing the paths and filenames as needed:

cat /home/hts/.xmltv/tv_grab_file.xmltv | socat – UNIX-CONNECT:/home/hts/.hts/tvheadend/epggrab/xmltv.sock

If typing “which nc” returned a path, then you can use this to import your guide data into Tvheadend using the socket method, replacing the paths and filenames as needed:

cat /home/hts/.xmltv/tv_grab_file.xmltv | nc -w 5 -U /home/hts/.hts/tvheadend/epggrab/xmltv.sock

If neither “which socat” nor “which nc” returned a path, but “which curl” did, then as a last resort you can use this to import your guide data into Tvheadend using the socket method, replacing the paths and filenames as needed:

cat /home/hts/.xmltv/tv_grab_file.xmltv | curl -d @- -m 5 -X POST –unix-socket /home/hts/.hts/tvheadend/epggrab/xmltv.sock

Whichever of the above three lines you use, note that you may need to run it as the hts user or you may encounter permissions or ownership issues. Replace “/home/hts/.xmltv/tv_grab_file.xmltv” with the path and filename of your xml file that contains your local listings, and replace “/home/hts/.hts/tvheadend/epggrab/xmltv.sock” with the path that was displayed in the “Path” field of the External XMLTV grabber in the “EPG Grabber Modules” tab (the one I mentioned that you would need to know a few paragraphs above).

If you can’t get this to work then all I can suggest for now is using the older method described earlier. But if you can get it to work, there are a few advantages to using a socket. For one thing, you can import your listings directly into Tvheadend in whatever script you may be running to get your listings – there will be no delay between the time you get your listings and the time they are imported into Tvheadend. And if you have to do an extra download of schedule data for some reason (say you add a new channel) the listings will be imported immediately.

After you get the schedule grabber working, all you need to do set up a cron job or scheduled task to a run your listings grabber (the zap2xml program, or whatever you use), or to run a bash or shell script that runs your listings grabber, once a day. Then if you are using the older method to import listings into Tvheadend as shown above, set it up to import the listings fifteen minutes after you have run the listings grabber (you could probably get by with a shorter interval, but why rush it – you want to make sure the listings grabber has completed its task before Tvheadend grabs the resulting file). If you are using the newer socket method, just include the “cat” command that uses socat, nc, or curl at the end of your bash or shell script.

NOTE: There are some people that find that for whatever reason, they cannot run the tv_grab_file script. This most often happens on systems or devices where bash is not installed (to determine whether that is the case, enter which bash at a Linux command prompt and if bash is installed it will display the path, typically /bin/bash). In such a case it may be possible to use a modified tv_grab_file script. For example, in my Review of the TBS MOI+, I showed a variation of that script that runs under ash (as provided in BusyBox), since the original bash script won’t work in that environment. But if you can’t do it that way, try using the newer socket method described above. That method is also discussed in this thread on the Kodi forum.

Note that after Tvheadend imports NEW channels, you MUST refresh the browser window before the new channels will appear in the EPG Source dropdowns under the Configuration | Channel / EPG | Channels tab. You may even need to close and re-open your browser. Failure to do this is probably the #1 reason people think Tvheadend has not imported the newly-added channels. You must select an EPG source for each channel before schedule data for that channel will be read into Tvheadend, which will happen on the next scheduled import of the TV schedule data.

NOTE: If you are using the older method (the tv-grab-file grabber), you can force an unscheduled read of TV schedule data in the newest versions of Tvheadend by clicking the “Re-run Internal EPG Grabbers” button under the Configuration | Channel / EPG | EPG Grabber tab. If you have an older version of Tvheadend that doesn’t have that button, you can accomplish the same thing by temporarily setting the Internal Grabber Module to “No grabber”, clicking the “Save Configuration” link, then changing the Internal Grabber Module back to “XMLTV: tv_grab_file …” and clicking the “Save Configuration” link again. If you are using the newer socket method, just re-run the “cat” command that uses socat, nc, or curl to re-import your data to Tvheadend.

When you are setting up your listings sources (channels) on whatever TV listings service you plan to use, if you don’t know which providers or stations carry a specific channel, you can look it up on Wikipedia, which will often tell you which providers and/or local stations carry that channel.

There are some national services that I will not name here, but that aren’t listed in the channel listings because they aren’t intended for viewing by home viewers. You need to get creative with those. For example, if you just happen to find a feed of the QXZ network, and you are smart enough to not blab about it all over creation so that the signal gets scrambled, you may be able to get listing data for at least prime time by grabbing the listings for the network owned “flagship” station in your time zone, which might be WQXZ on the east coast or KQXZ on the west coast. Of course the QXZ network is totally fictitious, but hopefully you get the idea.

If you spend a little time studying the XMLTV File format, you can even write your own scripts or programs to create “fake” XMLTV data for certain stations – I suppose you could even do it manually if you are a very patient and precise person.

There is one pitfall to all this, which was actually more of a problem with a grabber program that’s no longer useful due to changes in the underlying service, but I’ll mention it anyway just in case you ever run across it. The various schedule sources sometimes change channel ID numbers without any advance warning, particularly when you are grabbing terrestrial (over-the-air) channels. If you find that the schedule data for a particular channel “runs out” after a certain day, it’s probably because the ID numbers have changed. In Tvheadend, go to the Configuration | Channel / EPG | Channels tab, find the affected channel(s), and change the EPG Source using the dropdown – you will probably see both the old and new ID’s, usually one underneath the other. Uncheck the old one and check the new one, and the next time Tvheadend updates its EPG information, it should be okay (don’t forget to click the “Save” button before you leave the page!). By the way, in case you hadn’t figured it out from the previous paragraphs, that’s the place where you select the channel data to associate with a channel in the first place, but I will again note that if the EPG sources aren’t appearing in the dropdowns after Tvheadend has imported your xml file, then you may need to refresh the page, or close and re-open your browser.

EDIT: For those that have never set up zap2xml before, here is the general procedure. These are basically their instructions, but with some added comments to help clarify what needs to be done.

EDIT (January 2018): Unfortunately, Zap2it has changed the format of their web interface. The latest releases of zap2xml (dated 2018-01-10 and later) do still work with both Zap2it and TV Guide, but you will need to download the latest version, and be sure to read the 2018 release notes. The instructions have been changed to reflect the changes in the setup procedure. When using either Zap2it or TV Guide as your listings source, you definitely should set your favorite channels, but if using Zap2it you will want to note the new -8 option in the zap2xml release notes, while if using TV Guide I have found that favorites may not “take” on your first attempt to set them – once you log out and back in, they just might be gone, and you’ll just need to set them again and then log out and back in to see if they were really saved. The following instructions are for setting up zap2xml to use Zap2it’s listings, which I believe are slightly more accurate than TV Guide’s for non-prime time listings on certain channels..

1. Register your free Zap2it.com account

2. Click Change Provider to select a lineup (select country and input zipcode)

3. Optionally select your favorite channels (click the channel stars or click “Find Channels” link) to limit channel output

4. You may need to install the required supporting perl libraries (not needed with the Par-Packed Windows file) – COMMENT: I’d try step 5 first before you go adding any perl libraries, as they may already be present.

5. Run zap2xml with the userEmail and password parameters of your account – COMMENT: Watch the output and see it complains about missing Perl libraries. If so, see step 4. Here is how you might invoke it, assuming that you are running Tvheadend and that your Tvheadend user is “hts”:

./zap2xml.pl -u user@email.com -p yourpassword -o /home/hts/.xmltv/tv_grab_file.xmltv -c cache -8 -F -T -q

(Leave off the -q while testing, because it may suppress error messages that you’ll want to see, and leave off the -8 if you have not set any favorites, since the -8 option tells zap2xml to only output favorite channels, such as those you would see when clicking on the “Starred” button in Zap2it). This assumes that you have created a directory named “cache” (as a subdirectory off the directory where the zap2xml.pl script is saved) and that you have either already created a file /home/hts/.xmltv/tv_grab_file.xmltv (can be a zero byte dummy file to start) and made it world writeable, or else that you are always going to run the zap2xml.pl script as the Tvheadend user, in order to avoid file permissions issues. Of course, you must create the /home/hts/.xmltv directory if it doesn’t already exist. The point is that when you run the script and it tries to create the /home/hts/.xmltv/tv_grab_file.xmltv file, you don’t want it to fail due to file permissions or ownership issues. You can read about the options shown above, and others you may wish to use, on the zap2xml home page (in particular, you may want to use something like -d 12 to get 12 days of listings instead of the default 7). NOTE: Should you happen to be running OpenElec or LibreElec, which I most emphatically DO NOT RECOMMEND, zap2xml.pl will not work for you, in part because you may find it difficult or impossible to get to a command line, but also because Perl is not available. There is an alternative for OpenElec/LibreElec that is written in Python, but I have absolutely no experience with it.

6. Optionally set up a cron job/task scheduler task to run it every day – COMMENT: Keep in mind that when setting up a cron job, you must use full paths, and not any shortcuts such as “.” or “~” in the path. If you are using the older method that uses tv-grab-file, the cron job or task should be scheduled to run a few minutes BEFORE Tvheadend’s Internal Grabber is scheduled to run. So if Tvheadend’s grabber is set to run at 2:43 A.M., you may want to run your cron job that invokes zap2xml at 2:28 A.M., giving it 15 minutes to finish (which is, generally speaking, more than ample time for it to run to completion). If you are using the newer socket method, then your cron job should run a bash or shell script that first runs zap2xml, then the appropriate cat command to send the tv_grab_file.xmltv file to Tvheadend’s socket, as described in the first part of this article. Here is a very simplified example of such a script, which assumes that you (as the hts user — use sudo su hts to temporarily become the hts user) have created a directory named /home/hts/listings and placed the zap2xml.pl file in that directory:

cd /home/hts/listings
/home/hts/listings/zap2xml.pl -u user@email.com -p password -o /home/hts/.xmltv/tv_grab_file.xmltv -c cache -8 -F -T -q
cat /home/hts/.xmltv/tv_grab_file.xmltv | nc -w 5 -U /home/hts/.hts/tvheadend/epggrab/xmltv.sock

You would set up your cron job to run this script as the hts user (assuming that “hts” is the Tvheadend username on your system). If you’d prefer to run it in your user directory rather than the hts directory, and to run the cron job as your user rather than hts, you can do that by adjusting the paths in the script accordingly but then you may have to deal with a permissions/ownership issue when attempting to cat the file to the socket.

Whichever method you use, please configure your cron job or scheduler task to run at some odd random time (in other words, not right on an hour, half hour, or quarter hour mark, and don’t use the example times mentioned above) so that everyone isn’t clogging up the servers at once.
(End of edit.)

Particularly if you are running a PVR backend system and can receive channels from multiple satellites, or you live in an area where you can receive stations from multiple TV markets on your terrestrial TV antenna, you may need to set up multiple accounts to get listings from multiple TV markets. In that case, you will probably want to save all the listings from each market area into a separate, differently-named .xml file, and then when you have obtained them all, combine them into your main tv_grab_file.xmltv file. There are utilities and scripts that can help you combine XMLTV format files – see A few Linux utilities that are useful for manipulating XMLTV schedule files. Or if you are using the newer socket method, you probably could just cat each of the individual xml files into Tvheadend’s socket as they are downloaded.

I am painfully aware that there is nothing at all that is easy about this process, and it probably makes you wish that we had European-style free-to-air services, where EIT guide data is embedded right in the program stream (lucky Europeans!1). But since we don’t, I just wanted you to be aware that you don’t need to have a blank EPG in your Kodi Live TV section. There is definitely a learning curve to getting it all working, but the more you work with it the more you will understand how all the pieces fit together. You may never be able to get schedule data for every channel you can receive, and you’ll obviously never find it for “wild feeds” that come and go, but I’ve been able to populate the EPG grid for quite a few of the stations I’m able to receive on my dishes.

(By the way, if you want icons for the channels, at least in Kodi you have to load those to the frontend system – as far as I can tell, there’s no good way to put them in the Tvheadend backend server and have Kodi get them from there. So just create a directory, dump your channel icons there, rename them to EXACTLY match the channel names except for the extension – for example, if you have a channel named “Big Fart Channel”2 then the icon file name should be “Big Fart Channel.png”, or whatever the correct extension is – then in Kodi go to System | Live TV | Menu/OSD and modify the “Folder with channel icons” setting to point to the directory containing the icons. And yes, I’m aware that you can supposedly enter a path to a “User Icon” for each channel in TVheadend’s Configuration | Channel / EPG | Channels tab, but those are really intended to be the paths to “picons” supplied by TV channels in some other parts of the world. I’ve found that if you attempt to use those, more than likely you are going to slow down Kodi or cause some other undesirable effect, such as Kodi hanging when you attempt to quit Kodi. You are better off to store your channel icons on each of your devices that run Kodi.)

If you know of any software tools that would make this easier, or pick up any hints or tips that I have not mentioned, please feel free to post a comment (but be aware that I will not approve spam comments that promote commercial services. Also, I don’t need any praise – if you like this article, don’t tell me, tell your friends that have satellite dishes and that could possibly benefit from this information!).

Here are some possibly useful links:

zap2xml software and documentation.
tv-grab-file, a file used with Tvheadend to import xmltv format data
Zap2xml for ATSC in OpenELEC (Kodi forum thread)
Installing zap2it grabber in OpenELEC (YouTube Video)
Setup instructions and files for Synology NAS users (Tvheadend forum thread)
NextPVR – EPG Setup – XML/XMLTV EPG – Zap2it & Zap2xml (NextPVR forum thread)
Home DVR Tvheadend OTA EPG Setup (Part 2) (YouTube Video)

Note regarding the video in the previous link: It shows the basic setup, but using an older version of Tvheadend, and using the free version of the mc2xml listings grabber which stopped working in July, 2015. So, don’t use mc2xml, use zap2xml instead. Carefully read and follow the instructions at the top of that page on how to set up your Zap2it account, and also consider utilizing the tricks I mentioned earlier in this article.

If you found this article useful, you may like my followup article, A few Linux utilities that are useful for manipulating XMLTV schedule files.

1 Sure, the lucky Europeans get EIT guide data with their free-to-air channels, but at least we don’t have to pay a “telly tax” on each TV set we own, so there’s that!
2 Someone REALLY should start the “Big Fart Channel” – that would be a real gas! And with that, this article has really bottomed out. What do you mean, my puns stink?

I hear there’s a problem with playing satellite audio…

You may not realize it but when you run a backend such as TVHeadEnd with a satellite tuner card or device for the purpose of recording or viewing Free-To-Air signals in North America, and then use Kodi as your frontend, there’s a good chance you are not hearing the audio correctly on some channels, particularly if you have a multi-channel audio system (5.1 or better). The reason is that many of the uplinkers encode their audio in a specific manner that Kodi doesn’t understand.

This is not the same issue I wrote about in Fixing the audio on recorded programs from a certain network (which shall remain nameless), although that does show that different uplinkers encode their audio in a different ways. In fact, on “feeds” channels the audio can vary from program to program.

When you play a stream or recording in Kodi, if you display the on-screen controls or press INFO on your remote (depending on the skin you are using), you will often see an indicator of what type of audio Kodi thinks it is receiving. If you see indicators for both “Dolby Digital” and “5.1” then it is probably decoding the audio correctly. If you see anything else, such as “MP3” or “2.0”, then it may or may not be decoding the audio correctly, depending on how the source was encoded.

The strange thing is that if you have a suspect recording and you happen to have a machine that has the MythTV frontend software installed and you play the recording there, you may in fact hear the full 5.1 audio! Play the same recording using Kodi and you’ll hear audio on all channels, but your ears might tell you there’s something wrong.

Here’s the issue: 5.1 audio has 6 channels – Left Front, Right Front, Center Front, LFE (low frequency audio generally played through a subwoofer), Left Surround, and Right Surround. Kodi will play the first four of those through the proper channel in almost all cases, but it’s the Left and Right Surround channels that are mishandled. If you play the recording in the MythTV frontend, you will hear the Left Surround audio come from both the left front and left rear speakers, but with equal energy from both OR more audio from the rear. Same with the Right Surround, except that it will only come from the right front and rear speakers. However, when that same audio is played in Kodi, what sometimes happens is that any audio that should come from the Left or Right surrounds will be played at equal volume from all four Left and Right speakers, or if there is a difference in audio energy it will be biased more toward the front speakers! In other words, the Left and Right channel separation is totally lost for surround sounds.

So as you are sitting watching a recording or a live TV screen, you may notice that you are getting more than just 2-channel stereo, but it doesn’t sound quite right. Kodi will tell you it’s only 2.0 audio, and if you have a display on your receiver it may tell you the same thing. In fact it actually is 2-channel audio, viewed one way, but with the 5.1 channel information encoded within those two channels.

I’ve been reading various threads on this subject over the last few days, and it appears that in at least some cases what is happening is that the uplinkers are using Dolby Pro Logic II audio or some variation. To confuse matters further, in the audio communities this is sometimes referred to as just PLII audio.

It appears that the MythTV frontend has code that will decode this audio correctly, while Kodi does not. And you can’t even use ffmpeg to post-process a recording and convert the audio to a format that Kodi understands, because although ffmpeg has a PLII encoder, it does not have a PLII decoder.

This was a topic of recent discussion in a thread on the Kodi forum, although it appears that the developers didn’t show much interest in fixing the problem. There is a reference to a Dolby Pro Logic II test video on YouTube, which if you can figure out how to download and save, it will show you the problem in Kodi.

The thread above references a “ffmpeg user thread” – in my searching I found this archive of ffmpeg mailing list messages, and if you look at the ones with “5.1” in the subject, I suspect that is the thread being referenced, though I’m not entirely sure since the person who mentioned that thread in the Kodi forum did not include a link. In any case, reading those messages proved rather informative, even if ultimately nothing was ever resolved.

Normally when I present a problem, I like to also present a solution, but it this case it appears that for the time being there isn’t one. Programs that have this type of audio encoding will not play correctly in Kodi – it’s not that any audio will be missing, it just won’t be coming from the correct speakers, and some people may not even care about that (and if you only have two stereo speakers you definitely won’t care). If you happen to also have the MythTV frontend installed, you could play recordings in that, but in my experience some satellite recordings do not play all that well in the MythTV frontend – you may get all the audio channels coming from the correct speakers, but the audio itself may have microgaps and breakups. I suppose this in part depends on how good your HTPC hardware is; if you have a better HTPC than I do, then recordings may play fine in the MythTV frontend for you.

Most people will not want to install MythTV just to hear better audio, so that’s not really a viable solution for most readers. And MythTV has problems of its own, as I mentioned in What is the best backend software to use for a Free-To-Air satellite TV system?. So I guess the point of this article is to simply bring awareness of the problem. Maybe if enough people are aware that there is an issue, someone more knowledgeable than I will contribute code to the Kodi or ffmpeg projects that can play or convert this audio format correctly, or perhaps someone will develop a (hopefully free) standalone converter that can reformat the audio in the .ts files to something that Kodi understands.

I will note that some AV receivers may play this audio correctly IF you tell the receiver to use the correct decoder, and IF you have all the capabilities of your receiver enabled in Kodi’s System – Settings – Audio section (particularly “Dolby Digital (AC3) Capable Receiver” and “Dolby Digital Plus (E-AC3) Capable Receiver”, also be sure to “Enable Passthrough” so those settings will be used). For example, with Yamaha 5.1 receivers, you may find that using one of the Dolby or Neo:6 settings plays the audio of certain channels or recordings through the correct speakers, but selecting a “normal” mode does not. I imagine this is true of some other brand receivers as well, and may require that the receiver has HDMI connectors (this may not work if you are sending the audio to the receiver over a S/PDIF conection). I have found that the “Neo:6 Cinema” setting will play many types of encoding correctly, but feel free to try others. On such receivers, the “SUR. DECODE” button on the remote is your friend!

There is a followup article:
Why you’re not hearing full surround sound on some satellite channels – or maybe just think you aren’t

How to reduce the remote control reverse skip time in Kodi / XBMC

EDIT: This article is applicable to Kodi Helix, and to some versions of XBMC. Kodi Isengard implemented Skip steps, which should be used to make this type of adjustment.  The method shown below will probably not work in Isengard and newer versions of Kodi.

If you use Kodi / XBMC as your frontend for watching programs off of your satellite tuners, you might have found it a bit frustrating that although the left and right arrows on your remote control will let you skip forwards and backwards in the recording, both skips are set by default to 30 seconds. So if you skip ahead and find you have gone past the end of whatever it was you were trying to skip, and skip back once so you don’t miss anything, you may still have to endure as much as 29 seconds of something you don’t want to see. Most of the time, forward skips in 30 second increments work out well, but we’d prefer backward skips to be smaller, so we can target the start or resumption of whatever we actually want to watch more closely.

Here’s how to change the backward skip time to 7 seconds. This is known to work with MCE-compatible remotes, and may work with some others as well.

First, locate your Kodi/XBMC userdata folder – if you don’t know where it is, see this wiki page. Within that folder there should be a folder called keymaps, and inside that folder there should be a file called remote.xml – if it’s not already there, you’ll need to add it.

If the remote.xml file is not present or is empty, then create it if necessary and add these lines:


If a remote.xml file already exists then it will be necessary to integrate the above into the existing XML file, which will be easy for those who understand how XML files are structured.

After you have saved the remote.xml file, if Kodi/XBMC is running, completely shut it down and restart it.

If this doesn’t work, you may need to find Kodi’s (XBMC’s) master remote.xml file, which is in different locations depending on the operating system (for example, in many Linux-based systems it is at /usr/share/xbmc/system/keymaps – to find the path for your operating system, you can look at this page, and find your operating system and the path associated with special://xbmc, then append /system/keymaps to the end of that path) and examine the remote.xml file there to find instances of


You must then recreate that branch of the XML “tree” in your … /userdata/keymaps/remote.xml file, in a manner similar to that shown above, but change all instances to


Note you don’t need to duplicate anything from the master remote.xml file you are not changing – Kodi/XBMC will use the master remote.xml file for anything you don’t explicitly specify in your … /userdata/keymaps/remote.xml file. DO NOT change the master remote.xml file – for one thing, if you ever update Kodi/XBMC then your changes may (probably will) be overwritten.

For an additional example of changing remote control button mapping in Kodi/XBMC see Optimize remote mappings for users.

By the way, if you’d prefer that the reverse skip time were a bit longer or shorter, you can change that by adding or editing a file in the userdata folder named advancedsettings.xml – for information on creating the file see this page. If you create the file or if it is empty when you start, then all it needs to contain is this:

<smallstepbackseconds>7</smallstepbackseconds> <!– Length of the small skip back when playing a video –>

Just change the “7” to the number of seconds you’d prefer for the reverse skip time. If an advancedsettings.xml already exists, you’ll need to combine the above with the existing file.

What is the best backend software to use for a Free-To-Air satellite TV system?

If you use a program like Kodi to watch live or recorded television, you’re probably already familiar with the backend/fronted model. To put is simply, the backend is a server that communicates with the tuners, such as a tuner card or USB-connected tuner, or even a network tuner such as a HDHomeRun. The backend system can sit anywhere on the local network and communicates with one or many frontends. The backend handles streaming live programming, or recording it for viewing at a later time. In contrast, the frontend is run on a computer that often connects to a television set via a HDMI cable. It displays live or recorded programming, and usually gives the user a way to schedule programs for recording and perform some other administrative tasks. Some tasks can be performed from either the backend or frontend, some from the backend only, and some from the frontend only. But generally speaking, the frontend is what the user most commonly interacts with after the initial configuration is finished.

But remember, it is the backend that performs the crucial task of communicating with the tuners, which includes sending any control commands that might be necessary. So, for example, if you have any switches (DiSEqC or 22kHz tone) between the tuner and your LNB’s, the backend will control them. If you have a USALS or DiSEqC controlled motor, the backend will need to deal with that as well.

Unfortunately not all backends are equal when it comes to dealing with Free-To-Air satellite. For one thing, Free-To-Air in North America differs from the variety used in Europe and some other parts of the world. For a decent explanation of the differences, there is a Wikipedia article on Free-To-Air – just skip to the section on North America and read that entire section.

Most of the configuration information that you find online for backend software is intended for European users and other users outside of North America. So setting up a backend system on this side of the Atlantic can be a bit tricky, even if you do pick good software for the purpose.

Note that it is usually possible to run a backend and a frontend on the same computer – you don’t need two computers for the purpose, and if the backend computer has a HDMI output and is in a location close enough to your TV that you can connect a HDMI cable between the two, there’s generally no reason not to run a frontend program on the same system as the backend. Some software installers set up a backend and frontend by default.

In my time attempting to set up a working backend, I have gone through three pieces of software – MythTV (Mythbuntu), MediaPortal, and TVHeadEnd. MediaPortal runs under Windows and the other two under Linux. If you really want to avoid Linux for some reason, then MediaPortal is an okay choice, but it has several limitations that particularly affect North American users.

The biggest problem with MediaPortal is that it has a very different way of controlling DiSEqC and 22kHz tone switches, and in my experience, it doesn’t do either very well (especially DiSEqC switches). As long as I connected an LNB directly to a tuner, MediaPortal for the most part worked great. But if you have more LNBs than tuners and you want to use them all, and you have to try to use a DiSEqC or tone switch, then you might find that everything falls apart.

Under MediaPortal, with TBS tuner cards you need to install some non-official drivers if you want DiSEqC switching to work at all. And the way they enable control of 22kHz tone switches assumes that anything connected to the “tone off” side is a C-band LNB, and anything connected to the “tone on” side is a Ku-band LNB. There are weird and wonderful ways to work around this but the point is that you can’t just say that when you want to select a particular LNB, the tone needs to be on or off, as the case may be.

I also found that even if I avoided using 22kHz switches, I could not tune Ku-band frequencies except by lying to the program and assigning them equivalent C-band frequencies. I suspect this was partly caused by the non-official drivers, so the end result was that I could have either used DiSEqC switching or tone switching, but not both.

And getting it to scan in channels in the first place had been a real problem. I quote from the Wikipedia article I mentioned above:

Because many of these broadcasts are essentially point-to-point transmissions, the originators often do not follow any international standards when setting various identification fields in the data stream. This causes issues with receivers and software designed for use in other parts of the world, as they may assume that if a channel contains the same ID information as another channel, those are duplicate channels. This may be a valid assumption in other parts of the world, but is almost never valid for North American FTA signals. When such an assumption is made, during a “blind scan” the receiver or software will often fail to correctly insert one or more channels into its database, or it may overwrite previously scanned valid channels (including other channels on the same satellite) with invalid information picked up from another, more recently scanned channel. If the end user does not understand what is happening, they may assume that the receiver cannot receive certain channels or that it is defective, yet if the correct data for those channels can be manually entered, those channels may become receivable. This problem can be mitigated if receivers can be set to ignore channels that appear to be duplicates during a “blind scan”, except when such channels are on exactly the same satellite and same transponder frequency (as might occur if the user rescans a previously-scanned satellite).

MediaPortal is a VERY Euro-centric program, and it has the exact issue mentioned above. So it might scan one transponder correctly, but then overwrite the channels it scanned with other channels from a completely different satellite and/or frequency. So, you wind up having to enter a LOT of channels manually. I’m not saying that no other software has this issue, because there are a few other programs that behave the same way, but it was definitely a huge issue in MediaPortal.

So, every program has its weird quirks, but MediaPortal seemed to have more than its share of them. In the end, there were two things that really put me off about it. One was that it ran under Windows – for several reasons I need not go into here, I much prefer running Linux on a server, even though I am not a Linux expert by any stretch of the imagination. And, that I could never count on it to record reliably – for reasons I was never able to determine, it would sometimes fail to record a scheduled recording, or only record a short bit of a program and then just quit. Of course when that happens, the first thought everyone has is that it’s a hardware problem, but that same hardware worked just fine with TVHeadEnd.

On the plus side, I will say that the MediaPortal forums were among the most helpful I have ever seen. If you don’t plan on using DiSEqC or tone switching, or mind using workarounds, and you don’t have the issue of it stopping recordings in mid-program (which no one had a good answer for), then it appears that you can get an answer to almost any question you might have. It might not be an answer you particularly like, but at least it may provide some sort of a workaround to get you going again. I don’t want to oversell them, but there is a night and day difference in the attitude toward less experienced users in the MediaPortal forum vs. just about any forum for a piece of Linux-based software I’ve ever encountered. If you hate dealing with “Linux snobs” and people whose idea of “help” is telling you to read man pages and/or use Google, then stick with MediaPortal, because perhaps in time they will work out some of these issues and come up with some software that doesn’t cause so many problems for us North Americans.

Prior to trying MediaPortal I had tried Mythbuntu, which is a Linux distribution that includes MythTV, and saves you the trouble of installing the MythTV packages and supporting software. You just install Mythbuntu on your backend and you are supposedly good to go, with both the backend and frontend available. I actually have some experience with Mythbuntu, having used it with a HDHomeRun in the past to receive over-the-air television. So I thought setting up a new Mythbuntu installation would be a piece of cake. Boy, was I wrong.

One of the HUGE design flaws of Mythbuntu is that it appears that the backend and frontend versions must always match. That is, you can’t run version 0.25 on the backend and 0.27 on the frontend and expect it to work. Either the two will outright refuse to talk to each other, or only parts of the program will work. This is a problem because people rarely upgrade all their computers and devices at once. And if your distribution’s repository only offers a particular version of the frontend that doesn’t match what you are running on the backend, you might be kind of screwed, at least until you can figure out how to match up the versions again.

So I had to drop back a couple versions of Mythbuntu to match what my frontends were running, and I’m not sure if that caused me problems or not, but the end result was that I could never get Mythbuntu to actually work. When it did a scan it would appear to find some (but not all) the satellite signals that were up there, but there was no way it would play any of them. You could click all day and just nothing. That, coupled with some information I read in various forum threads as I was trying to figure out the crux of the problem, led me to think I was pursuing a fool’s errand. The impression I got was that MythTV USED to work for receiving satellite programming at some time in the past, but something is broken now so that in recent versions it doesn’t work anymore. Another issue was that even when people had been able to get it to work, it would not receive certain high-bitrate signals such as 4:2:2 format signals.

I am only willing to beat my head against a brick wall for just so long, so that’s why I moved on to MediaPortal. I’m not saying that no one could ever get MythTV to work in this application, but it appeared that other people who seemed much more familiar with Linux than I had tried and failed. I got the distinct impression that no one in North America has been able to use a recent version of Mythbuntu for this purpose with satisfactory results, or at least not anyone who was willing to speak up in the various forums.

The real winner of the three is TVHeadEnd. Now, TVHeadEnd is a bit different than the other two in that as the name implies, it is backend software only – there is no specific TVHeadEnd frontend. I believe that most TVHeadEnd users use Kodi as their frontend software, although there are undoubtedly other choices.

I covered some aspects of TVHeadEnd configuration in my Review of the TBS MOI+ DVB S/S2 Satellite TV Linux Server (see the section on Configuring TVHeadEnd) but I just want to emphasize one point – many of the options in TVHeadEnd are not exposed until they are needed. So if you just install the software and start looking for things such as “Where do I tell it what type of LNB I have?” or “How do I specify that I am using a DiSEqC switch and/or 22kHz tone switch?”, you won’t see those options until you have set other options that expose those settings. The aforementioned review shows examples of this.

Because of that, some users think that TVHeadEnd doesn’t support DVB-S/S2 satellite tuners very well, when the truth is that it does a much better job of it than either Mythbuntu or MediaPortal. I’ve never seen TVHeadEnd fail to record a scheduled program unless there was some obvious reason, such as no available tuner due to a scheduling conflict, or a problem with an LNB or DiSEqC switch. or the program starting in the afternoon right in the middle of a solar outage period.

Where TVHeadEnd does fall down a bit is in scanning satellites and transponders. What you are probably going to find is that you will need to add a few of your desired transponders manually, and maybe even the occasional channel or two. Once you have used TVHeadEnd a few times, this all becomes fairly intuitive, but it can really confuse new users. On the plus side, it appears that TVHeadEnd is still under active development, so some of the things that were confusing in earlier versions are less so in more recent ones, and sometimes new functionality is added. For example, in my review mentioned above, I wrote:

I should mention that TVHeadEnd does have one glaring deficiency – if you don’t have schedule data available for a particular channel, then to the best of my knowledge you cannot set up a recurring recording on that channel. You can tell it to record once at a specific time on a specific channel, but without schedule data there is no way to specify that you want to record that same channel at the same time every day or every week.

And while that was true for the version of TVHeadEnd included with the TBS MOI+ that I received for review, it’s something that has been addressed in newer versions of TVHeadEnd.

Now you can specify the duration of recordings and day(s) of the week to record.

Now you can specify the duration of recordings and day(s) of the week to record.

(Note that for most channels you should be able to obtain free schedule data. Don’t subscribe to a pay service for schedule data, just follow these hints to set up your program guide in TVHeadEnd).

I am not saying that any of these programs are perfect, but once you get past the initial hurdle of getting your mind wrapped around how TVHeadEnd operates, it all clicks into place. And of the three, TVHeadEnd has so far proven to be the most trouble free and the most reliable of the bunch.

One downside of using any Linux-based backend is that you may need to install drivers, either to get your tuner to work at all, or to get it to work at peak efficiency. And if you are used to installing drivers in Windows, where it’s more or less a point-and-click type of operation, you may find that the procedure for doing it under Linux is a bit off-putting. For certain tuner brands, it is not very difficult if you can follow instructions (for example, this page explains how to do a TBS driver install under Ubuntu 11.10, 12.04, 12.10 and 13.04 (TBS6921 and other)). But not everyone finds it easy; sometimes there are entire forum threads about driver installation. If you’ve never touched Linux before in your life, then you may want to seek help from an experienced Linux user. Make sure you watch how it’s done and take notes, because you will likely need to do it again if you ever upgrade the Linux kernel on your system (and Ubuntu loves to push kernel updates every so often, so don’t just blindly accept a software update 10 minutes before you have something scheduled to record!). This bash script can help simplify the task of rebuilding the TBS drivers after a kernel update.

Also, you need to be aware of the limitations of various tuners. Some older DVB-S tuners cannot receive DVB-S2 signals, and some DVB-S2 cards cannot receive 16APSK signals (if you really want to receive one of the handful of 16APSK signals on North American satellites, and you are considering the purchase of a TBS tuner card, it is strongly recommended that you purchase one of the “professional” grade models). Some tuner brands do not support their tuners with updated drivers, meaning that people may recommend you use some unofficial driver from a questionable source. Or, if you want to use more than one tuner card from the same manufacturer or of the same model in the same backend system, you may find that’s impossible because the driver programmers apparently never anticipated that someone might like their tuners enough to want to buy a second one (I guess that lets you know what they think of their product). The may be certain tuner models that will not work at all under Linux (you can find lists of DVB-S2 tuners known to work in Linux by following the links on this page, but just because a particular tuner isn’t mentioned does not necessarily mean it won’t work – it may just be that no one ever bothered to add it to the page). Problems caused by tuners or by tuner drivers will likely affect any backend software you might choose to use.

With TBS tuner cards in particular, you need to be careful that they are not sharing interrupts (IRQs) with other hardware devices in your system. Also, if you have a newer model TBS card and find that it will not scan muxes to find channels, try this. If you find that many of your recordings have bad timing information, and it’s not due to a weak signal, it is sometimes possible to fix that by changing a setting in TVHeadEnd.

If you start asking around about which tuner you should purchase, please keep in mind that the “Linux snobs” (the type that will only point you to man pages and Google when you need help) may prefer a particular piece of hardware, but only because they’ve had enough Linux experience to work around any issues they encounter. Just because they could get a particular tuner to sing and dance doesn’t mean that you can, unless you have a similar level of Linux experience. If you ask in a Linux forum which tuner is best, and make a purchasing decision based on the responses you receive, and you are relatively inexperienced with Linux (or you are simply not a Linux devotee), there is a real good chance that you that you will not be happy with your purchase. One tip, no matter how much they may be recommended by a particular user or group of users, avoid older PCI cards – any tuner cards you purchase now should be PCIe (note the “e”) cards.

Unfortunately, not all PCIe tuner cards are compatible with all motherboards. I’ve had good luck with a MSI motherboard (specifically the model B85M-G43, although I’m sure many other models will work equally well), but when I had tried using the same tuner card in a particular Gigabyte motherboard it would not work. A real Linux expert perhaps could have figured out the reason why, but since I’m not a Linux expert it was a lot easier to just try a different motherboard. Had I known about this thread in the TBS forum, I might have tried the suggestion there to see if I could get it to work.

There are also USB-connected external tuners, and while some users have had no problems using these, several others have reported problems with overheating leading to premature failure. Also, with certain USB-connected tuners there is an extra step of installing firmware, in addition to the drivers, at least when using Linux as the operating system. I’ve never used a USB-connected tuner, but if you decide to try one, I’d at least keep it in a well-ventilated place.

Now, I don’t mean to say that the backend choices I have mentioned above are your only three options. There are, in fact, other programs that that can act as a backend program, but I just haven’t tried them. The TBS MOI+ that I reviewed offered a choice of TVHeadEnd or VDR. I did look at VDR once, for all of about half an hour, but could not figure out how to do anything in it and just generally didn’t care for it much. Perhaps if I’d had clear instructions on how to proceed, I might have liked it more, but I got the feeling that it’s a program that very few people in North America use. Which doesn’t necessarily make it good or bad, it just means that when you have problems, there are going to be a lot fewer people around that might be able to help you. If you prefer running Windows rather than Linux, but don’t care for MediaPortal, NextPVR is another somewhat popular choice, but I have no personal experience with it so I cannot offer any opinion on it.