Latency Measurement, Part 2: Resampler Tuning

Written 2022-03-30

Tags:Latency CorrelationCodes Video BarkerCodes Correlation 

One of the challenges with time-aligning correlation codes blinked out by a computer monitor and into a camera is that the sample-rates(FPS) are usually not the same, and even if close, they're slightly different.

On my first pass, my stopwatch started when the frame went out, and stopped when the next frame in decreased in correlation strength(also a frame late). This limits the timing resolution to 1 input frame time. Adding quadratic interpolation to compute the peak correlation strength in between frames helped enormously. Adding an anti-aliasing filter when resampling the correlation code used at the camera helps too.

In the following plot, Lag curve is the old approach, and Lag2 curve include parabolic peak interpolation and resampler filter. Lag1 is roughly 33ms(one input frame time) later/higher than Lag2 because it always picks the first decreasing correlation frame, but Lag2 picks a closer point in time.

glBarkerLagPlot_LpfParabolic

While there is still a significant offset error in the below graph, the variation is down to around 4ms now, which we can see in the following histogram(horizontal scale width same as previous post below).

glBarkerLagHist_LpfParabolic

Automatic Video Latency Measurement, Part 1

Written 2022-03-22

Tags:Latency CorrelationCodes Video BarkerCodes Correlation 

Problem

I want an automatic method for measuring video latency of a wireless video transmission system. As always, I'm willing to spend far more time automating than it would save one person.

A common current approach is point the wireless camera at a moving target like an analog ticking clock or video of counters or another moving pattern, then use an additional camera to record both the wireless video link display along with the reference target. One of my favorite targets was a chain of indicators on lego gears driven at high speed, each indicator gear reducing the ratio for the next so it had an enormous repeatition interval.

Overall system idea

If we can run some code on the video transmission system, render some images on its display, and analyze frames coming in, we can make the thing measure itself. Once that's measured, we can insert another video link if desired to characterize it.

Video Target Design

To support cameras with auto-exposure and auto-white balance, we need a target with a somewhat stable average brightness and a spread of colors for white-balance.

After a few tries here's what I use:

glBarkerLagCalTarget

The inner red/green/blue/grey squares are fixed while the outer corners blink together and edges blink together opposite the corners. In this way, there's always plenty of full brightness and darkness in every frame, and a little color.

Short intro to Barker Coding

Barker codes are short binary(in the sense of two discrete values, -1 and 1, not 0 and 1) patterns of numbers that have a few useful properties:

This means that if we transmit a Barker code by modulating the display by blinking similarly to programming a Timex Datalink, we can measure when an event occurs by sending out a Barker code then listening for the same Barker code. This general approach of marking a transmission with a correlation code of favorable properties is often used in communications to mark the start of a packet or other timing sensitive information.

Go here to read more: https://en.wikipedia.org/wiki/Barker_code.

Overview

Here's what it looks like - left side is the output target, right side is the camera preview with thin blue alignment rectangles to help you see where to aim it for the corner boxes. Program supports automatic measurements and one-shot(default).

glBarkerLag

Initial Results

Here's a recording of latency over 71 latency measurements from a 60Hz Thinkpad LCD to a 30Hz PS3 Eye camera. Variation is within 20ms and an input frame is at most 33ms here. Computing at sub-frame offsets in time is the next step for improving this.

glBarkerLagPlot

Here it is again in a histogram:

glBarkerLagHist

Implementation Challenges and Tradeoffs

Operating System Support

Windows has bad OS timing resolution. VSYNC on Linux is hard. I ended up using Linux.

Video Output

Video output seemed straightforward except that at first I used OpenCV which doesn't directly support VSYNC which is needed to measure output FPS, so I ported the video output to OpenGL, then reconfigured my video driver to enable VSYNC, then reconfigured my xserver to enable VSYNC. Year of the Linux desktop and all.

Video Input

Using OpenCV's video stream blocks until a frame is ready, which can often take longer than an output frame depending on input and output frame rates, causes the output stream to fail to draw each frame - this is important for correctly transmitting a code.

The common solution is to use one thread for the camera and one for the display works well, though the startup code is complicated as we use camera resolution to decide display window resolution, and some of the initialization code on each thread seems to cause the other thread to stutter a few frames until we get going - maybe it's Python's GIL?

scan-in/scan-out synchro

When I first got this working with a USB webcam, each run would have different average latency - not wildly different, always within 1 input frame time. I suspect this is due to variation in when the camera starts its scanout vs when the display starts its scan-in. Also, the camera and LCD VSYNCs are not synchronized, so they do tend to drift over time.

Possible Future Improvements

TRS to XLR Adapter Noise Floor

Written 2021-12-30

Tags:microphone audio XLR 

Today I take a look at the noise floor of my Tascam DR-44WL with a couple phantom power XLR to plug-in power TRRS microphone adapters.

Test Setup

Tascam DR-44WL set away from other electronics and recorded with no adapter, and again with each adapter in XLR port 1 with 48V phantom power. No input termination was used. The recorder was set for +15dB gain, the most sensitive setting.

Analysis and Results

All three recordings were run through a 16k point spectrum analysis in Audacity, then each spectrum exported and combined in OpenOffice Calc. This first plot has a linear frequency horizontal axis.

16kFFT_noise_floor_15dB_boost

The first thing I noticed was a slight peak around 34.5KHz, but I would never hear it, and it's present without the adapters so it's either inside the recorder or around my home. Also note that the orange(Rode) and yellow(Movo) curves overlap nearly exactly, and are both higher than the blue curve(no adapter).

16kFFT_noise_floor_15dB_boost_logfreq

If we rescale the horizontal axis with a log scale, we can see the effect of the added noise of the adapters at 20Hz can be 20dB worse than without one, but the difference rolls off as we go up in frequency. Really though, these are likely to be combined with a low-cut filter for voice recording.

Tomi Engdahl has a good write-up on plug-in-power and a few different designs for how these adapters work, if you scroll to "Balanced electret microphone circuit" it is quite similar to what Zach Poff found.

Initially, I was concerned these adapters were going to add a lot of wideband noise, as that's what Zener diodes emit. However, there's a little more going on here. The actual impact to noise floor is both lower than I had suspected, but still a little disappointing that it reaches up into the voice band.

First, the tantalum capacitor in parallel with the Zener diode does a lot to filter out noise. The choice of tantalum here may be important, as electrolytic capacitors tend to have higher ESR than tantalum, and more ESR would limit ability to filter noise here.

Second, the Zener diode is powered from the XLR audio+ and audio- lines through a pair of resistors. Because XLR is differential, and similar amounts of Zener noise should flow back through both resistors, most of the Zener noise seen through the XLR should be common-mode and thus easy to reject.

TRS to XLR Adapter Frequency Response

Written 2021-12-30

Tags:microphone audio XLR 

I've got a couple TRS microphone to phantom power XLR adapters, one Rode VXLR+ and Move F-XLR Pro. The Rode has a better housing and interoperability with my equipment, so it's already my favorite, but I was curious about their frequency responses and if there were any band limits.

Test Setup

sweep_setup
  1. Rigol DG1022 DDS waveform generator emitting 100mVPP sine waves, 50Ohm source
  2. 6uF capacitor, 1kOhm resistor in series
  3. TRRS pigtail, select leads according to TRS/XLR adapter under test
  4. Tascam DR-44WL audio recorder in 96KHz 24bit WAV mode, using first XLR input
  5. Step through each frequency, dwelling several seconds

Analysis

For each tone, I selected the relevant segment in Audacity's spectrogram view:

audacity_spectrogram

Once a tone is selected, I used analysis->plot spectrum to bring up an FFT of only the selected interval. I used a Hann window and 65536 points, though this image shows a 4096 point, as the 65536 point FFT pixels are smaller than display pixels and become hard to see.

frequency_analysis

Then I selected the peak frequency and amplitude and saved to a spreadsheet.

Because of the DDS generator, I did not try to measure SNR. SNR through these adapters is interesting, as they use a Zener diode and resistors to conver the 48V phantom power to 3-5v microphone power, but in doing so, emit some noise or hiss, and the selection of Zener diode becomes quite important. Kamil's tech tips found the Movo to be around 7dB noisier than the Rode. I might look at noise floor separately later.

Results

The Rode and Movo adapters have eerily similar response curves. It turns out that they are largely the same design with a slightly different wiring, so this actually makes sense. Zach Poff has an explanation and reverse engineered schematic here.

frequency_response_rode_movo

The low end is fine for speech and the high end actually extends a bit past nyquist(48KHz here) and aliases back in the spectrogram - I'm not sure if the adapters are limiting it or an anti-aliasing filter inside the Tascam recorder. Make no mistake, this roll-off is necessary and important, but if I were making my own adapter, it might make sense to move the cut-off even lower

Error Analysis

Debian Bullseye + ZFS on Lacie 5Big NAS Pro

Written 2021-11-26

Tags:5big NAS Debian Lacie 

I've added a SanDisk Ultra Fit USB disk to my 5Big NAS Pro for the purposes of setting up Debian with ZFS as a NAS. TrueNAS is another option, but suggests a minimum of 16GB of RAM, while the processor in the Lacie 5Big NAS Pro is limited to 4GB. ZFS does work better with more available memory, but I don't plan to use deduplication, and only would like to sustain a 1gbps ethernet link.

System Configuration

5Big NAS Pro, unlike Lacie's earlier, ARM-based systems, has an Intel Atom processor, DDR3 SO-DIMM RAM, and a traditional BIOS accessible with a keyboard and VGA monitor. For my system, I added an internal USB stick for the OS, but this is not required - you could also use a SATA disk, though I wanted to reserve all of those for storage. For now you'll need to set the BIOS to boot from an external USB stick over any SATA or internal USB disks.

Base Installation

I used the ISO from here: https://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/current/amd64/iso-cd/ and then wrote it to a USB thumbdrive, inserted it to the NAS, booted and installed. Notably, though the BIOS appears to support EFI, I couldn't get it to boot Debian, and so re-installed with GRUB and no EFI partition.

Apt Configuration

Add or update the following entries to /etc/apt/sources.list

    Add backports: deb http://deb.debian.org/debian/ bullseye-backports main contrib non-free
    Add contrib non-free to: deb http://deb.debian.org/debian/ bullseye main contrib non-free
ZFS is in bullseye-backports and we need bulleye's contrib and non-free components to update the CPU microcode. Once done, run 'apt update'.

Additional Packages

You may find the additional packages useful

    sudo
    openssh-server #remote access
    htop #better system monitor
    nload #network monitor
    fatrace #watch for unexpected disk accesses
    powertop #power monitoring and configuration
    intel-microcode #ZFS guide suggests running up to date microcode
    linux-headers-amd64 #will be needed for installing ZFS
    hdparm #used to set spindown for power savings
    samba #used for windows file sharing
    nfs-kernel-server #used for UNIX-like file sharing
    i2c-tools #needed for scanning the motherboard

Debian recommended installing ZFS by specifying the target release like so:

apt install -t bullseye-backports zfsutils-linux

Configuring Spindown

Note, this is a point of contention among many, and it is easy to end up with a system where your disks spin up and down too often. Keeping the disks up limits latency and wear and tear, but does cost power. In my case, the NAS lives in a small office with poor ventilation, and is not accessed often, so I enabled spindown

Edit /etc/hdparm.conf and uncomment or adjust(it means N * 5 seconds):

spindown_time = 24

Configure Swappiness

Because I placed a swap partition on a USB drive, I lowered the swappiness from default to 10. To do so, add a line(or file) to /etc/sysctl.d/local.conf with

vm.swappiness = 10

Setting up ZFS

At this point, pick up at https://wiki.debian.org/ZFS#Creating_the_Pool and continue there.

Misc Powersaving Tasks

Once installed, powertop can provide guidance on power tuning. If disks support it, ALPM can save some power as well. fatrace can help identify any periodic disk accesses as well.

zfs-auto-snapshot

The Debian package zfs-auto-snapshot will set up cron jobs to automatically create snapshots periodically. Note that if you enabled spindown, snapshots will spin up the disks. Specifically /etc/cron.d/zfs-auto-snapshot will snapshot every 15 minutes, and may need to be removed. Snapshots are pretty cheap in ZFS, but I kept only the weekly and monthly, as this is a low-write device.

Older