Time multiplexed WAV Colour Standard

[AncientBrit]

The club already has a proposed colour standard utilising the WAV file format. The CCNC standard allows the colour information to be embedded in the lower byte of a double byte word with the luminance occupying the majority of the upper byte. When this is decoded by a WAV codec the predominant analogue output is a compatible Y signal which may be displayed on a non modified NBTV viewer. The colour information is not decoded but appears as very low level “noise”.
For full colour decoding access to the digital signal is necessary to achieve the necessary precise separation of the two bytes producing the full RGB display signal.

The attached zipped file contains an alternative suggested standard but instead utilises time compression to contain the Y signal and its associated U and V colour difference signals. The Y signal is time compressed and occupies the first half of the NBTV line, the U and V signals are further time compressed to fit into the remaining half of the line time. It is immediately obvious that the signal whilst having the same timing format as a mono NBTV signal (32 lines, 2.5mS active line) when displayed on a standard televisor will produce a squashed height luminance pix and two unrecognisable colour difference signals.

However if it is accepted that for colour some form of interface has to be present between WAV player and display device this is no disadvantage. The interface can be simply connected straight to the analogue output of the player. It requires no access to the digital signal within the player. The interface needs modest storage for all it has to do is to store the previous NBTV line of information, time expand the Y signal by 2 and matrix this with the U and V signals time expanded by 4 times.

A WAV file running at 44.1 or 48 kHz sampling will hopefully yield a recovered audio bandwidth of around 20 kHz. The time expanded Y signal will have an audio bandwidth of around half this (10kHz) whilst the U and V signals will have a bandwidth of ¼, say 5 kHz. The ratios have been chosen to tie in with classic TV theory that predicts a chroma bandwidth of half luma is perfectly acceptable.
Astute readers will recognise the proposal as an NBTV adaptation of MAC (multiplexed analogue components).

One of the primary advantages is that only a single channel is needed to convey the full signal and the sound card can be standard issue.
There is no messy subcarrier or crystal locked oscillator.
And lowering the analogue system bandwidth will just lead to a more “soggy” picture.
The second channel is available for audio, and 8bit resolution is more than adequate leading to reasonable file sizes.

The disadvantages are that interfaces are needed at input and output but at least these are handling audio signals (although DSP would be used within the interfaces)., and the multiplexed signal is not monochrome compatible.

As a first step in demonstrating the concept, software was developed to playback WAV files encoded to the multiplexed standard. The majority of the active line is devoted to Y, U and V signals with a single sample being allocated to line sync. This may prove to be too narrow for accurate recovery after LPF in the sound card. And it remains to be seen what a 10kHz bandwidth limited signal actually will look like on a televisor.

The signals decoded and displayed by the programme on the PC do not have to pass through any analogue filtering but a small amount of interpolation has been added to improve the screen appearance. Also at the moment synching is hard locked to the 48kHz sample rate (120 samples per NBTV line) and only an odd/even flag is added when interlace has been recorded to synch up the frames.
Note that interlace must be selected manually when needed, at the moment this is not an auto function.

The signals were generated in another programme I wrote that down samples a WEB camera.
This programme produces 64pixels x 32lines x 37.5 sequential RGB frames per sec and another utility then converts the RGB frames to the YUV 12.5Hz format that is utilised here.

At the moment this is a concept only. A full working model would use 44.1kHz sampling so the files could be laid down on a CD. And pix synching would need to be refined.

The source files are included as well as the VB6 compiled .exe.
The code contained in NBTVplayer.frm may be viewed with a simple text viewer.
If you use any of the code an acknowledgment would be nice.

The program has run on Vista and Widows XP


G.Lewis April 2008

[Steve Anderson]

Graham, in a word, excellent!

It's certainly the format to go for as opposed to other proposals, it's not directly mono compatable, but it's a comprimise I'm willing to accept.

The application of YUV encoding in the method you have done here only doubles the required bandwidth, not triples it.

Graham, well done. When I get to colour it's the format I will use athough with different line and frame rates.

It runs fine on XP Pro SP1 here.

Steve A.

[AncientBrit]

Thanks Steve, nice to get some feedback,

Cheers,

Graham

[DrZarkov]

It does not work here with Windows XP SP2:

[Viewmaster]

Wonderful colour. Is that you? Would like to see some landscapes too.
Works here on XP home edition SP2.
Albert.

[DrZarkov]

It was a DirectX problem, which I've solved. It's now working fine here, too. The colours are looking good, but I would like to see some testcards, if possible. The black and white picture on the right is looking o.k., too.

Well done!

[AncientBrit]

> Dr.Z

Glad to hear you got it working. In fact I sat down last night to re-write the prog so it did not use the comdlg32.ocx (common dialogue ocx).
But you won't need that now. But I've attached it anyway

If I produce some electronic cards I'll have to think about "filtering" them so they represent a "real" image.

Alternatively I could capture some images from the web cam. but again I have to think through how I correctly down sample. At the moment I just average surrounding pixels.

Perhaps Gary could suggest some (simple) algorithms!


>Viewmaster. Yep it's me


Regards,


Graham

[gary]

Perhaps Gary could suggest some (simple) algorithms!
Graham

Sorry Graham, I am not exactly sure what you are requiring the said algorithm to do, can you elaborate?

I use a number of techniques in my software including image resizing and sample rate conversion. For instance, in the Video2NBTV app, after the video is cropped to the aspect ratio, it is resized, if necessary, to a size that creates an integer number of samples in the line direction (i.e. in the case of 32 line the height of the image is changed to 110.25 * 4 = 441), sync is added and then the serialised signal is low passed filtered to 10kHz before decimating by 4 to the required sample rate. In the player the reverse procedure is applied.

[AncientBrit]

>Gary

The problem I have is that my maths is useless so any decimation or interpolation I carry out is largely by trial and error.

I would be interested in a general algorithm for low pass filtering (decimating) when I am downsampling a web cam image.

Taking a practical example with the web cam, I take a pixel, give it an arithmetic weight of say 3 and add in nearby pixels each with a weighting of 1.
Sum the whole lot and divide by 7.

This certainly gives me some form of average but I would like to refine the technique.

Any pointers, suggested reading etc ("Filters for those with limited Maths" etc) greatly appreciated.


Kind Regards,


Graham

[gary]

>Gary

The problem I have is that my maths is useless so any decimation or interpolation I carry out is largely by trial and error.

I would be interested in a general algorithm for low pass filtering (decimating) when I am downsampling a web cam image.

Taking a practical example with the web cam, I take a pixel, give it an arithmetic weight of say 3 and add in nearby pixels each with a weighting of 1.
Sum the whole lot and divide by 7.

This certainly gives me some form of average but I would like to refine the technique.

Any pointers, suggested reading etc ("Filters for those with limited Maths" etc) greatly appreciated.


Kind Regards,


Graham

Aah but thats what I mean, are you down sampling for image resizing or down sampling for conversion into an NBTV audio stream?, surprisingly there is a difference in approach.

If image resizing, the three main approaches I use are nearest pixel (more or less your averaging approach), bilinear, and bicubic - although recently I have become interested in Lanczos which is a windowed sinc (not sync!) approach, and, of course, there are many other filters as well.

The basic difference in the three I mention is the trade-off between the reduction in pixellation and the amount of blurring.

Now, the question is, do you want to re-invent the wheel and code up one or all of these yourself (not particularly difficult) or use a freely available library? I recommend the latter because you get a lot of additional functionality as well, but if you are doing it as a purely intellectual exercise then maybe you would prefer to do your own, in any case these libraries include source code so tend to be a source of instruction as well. The other reason for doing your own is for speed, as I have done for The Big Picture!, but that is only because I wanted to do full screen video and that meant DirectX, or, in my case, assembly language.

If you decide on using a library you may need a DLL (all of the libraries I am familiar with, e.g. CxImage, are C++ oriented because that is what I use). If you wish to go with a library I am familiar with I could easily knock up a DLL for you so you can use it from VB if necessary.

If you want to code it yourself I can certainly provide you with algorithms.

On the other hand if we are talking about down sampling to audio then this becomes more of a standard sample rate conversion problem, indeed this has been my approach for creation and playback usually to and from 48 kHz and 44.1 kHz. In this case there is, as far as I am aware (as I have alluded to in another thread) no generic solution other than the usual sample rate conversion approach of interpolation/decimation and filtering. The problem is is that a filter has to be designed for each different resampling requirement. Also, it is always necessary to find an integer multiple of the two rates - not always easy.

Interpolation is merely adding zeroes between samples before FIR filtering to the original rates cut-off frequency, decimation is merely throwing away samples after FIR filtering. So the major task is to come up with a FIR filter. This I have discovered to be a bit of an art since you want the frequency restricting (anti-aliasing) performance of audio filters with smoothing (anti-ringing) performance of video filters. This becomes a compromise, and, as Klaas has shown me with his approach for he and Vic's colour system, can be relatively simply implemented with a 15 coefficient FIR filter designed by a Cosine windowed Sinc function, and which has a much slower roll-off than you would normally use for audio.
(It can also be done in a more technically correct way by using a very high order filter that incorporates brick wall roll-off with very low ripple but this runs into CPU performance problems when coded by hand - unless in assembly - and appears to be unnecessary).

In the case of the above mentioned colour system I resize the source video to 288 x 441 pixels using image cropping and resizing techniques, FIR filter to 10kHz, and decimate to 44.1kHz whilst serialising to audio.

I can elaborate on these methods if they relate to what you are wanting to do.

Cheers,

Gary

[AncientBrit]

>Gary,

I hadn't realised what a can of worms I was opening!!

There's more to this than I had imagined.

In the case of the above mentioned colour system I resize the source video to 288 x 441 pixels using image cropping and resizing techniques, FIR filter to 10kHz, and decimate to 44.1kHz whilst serialising to audio.

As a start if you could give me more info on the 10kHz FIR and decimation process (and pseudo code if available) it would tie in with my latest task. At the moment I soften the cropped Web image by a simple averaging process plus judicious de-focusing of the camera!

I too have found problems with lack of processing speed but VB was never known for speed (sorry C, C# etc is a black art for me!!).

I have used Safe Arrays in the past to improve plotting to screen, but pixel value computation still remains an issue.

The DLL would be extremely helpful but you of course you will need to know what to include and at the moment I can't advise since I'm feeling my way forward.

Thanks you for your prompt response and very helpful comments,


Kind regards,

Graham

[AncientBrit]

>Viewmaster

More material.
Not a landscape, more a garden-scape.

Handheld WEB cam so a bit of camera shake.
I haven't got the filtering quite right in the downsampling so the interlaced image is a bit blocky.

Regards,

Graham

[Viewmaster]

>Viewmaster

More material.
Not a landscape, more a garden-scape.

Handheld WEB cam so a bit of camera shake.
I haven't got the filtering quite right in the downsampling so the interlaced image is a bit blocky.

But colour is good. The flower pot/chair/paving all look correct.
Just like watching 'Gardener's World.'
It really is a very natural looking colour. Neither weak nor garish.
Albert.

[AncientBrit]

Thanks Viewmaster.

Graham

[DrZarkov]

It works perfectly (both, the new version of the software, and the garden-video). The colours are looking natural, and the black-and white is still acceptable good.