GNU Radio: Some More Tools for Offline Processing In MATLAB

In my last post I talked about how data can be stored and read in MATLAB for analysis. The only downside is that these data files do not include any information about timing relative to other blocks. GNU Radio has a built-in functionality with stream tags to keep track of timing information. This can be useful for working with packetized data, but currently there isn’t any way to save those tags along with data to look at later. I’ve been working on a few blocks that might be able to help with this.

Bursty Data & Stream Tags

Originally GNU Radio was great for continuous streaming of samples, but lacked the ability to pass ancillary information like meta data, packet lengths, and the like that would help with decoding bursty data sent in packets. However, in recent years there has been quite a bit of work done to improve this.

One of the features that has helped extend the project’s functionality is Stream Tags. These stream tags operate as a separate data stream to pass information along to blocks downstream in the flow graph. These tags are associated with a certain sample in the data stream and that timing relation is preserved as the tags propagate through the remaining blocks. For instance, there are several blocks in GNU Radio that expect the first sample of a packet to have a tag containing the length of the packet. The block can then do its processing based on packet length.

Stream Tags are Polymorphic Types, a data type that can be used as a general container for data, be it vectors, strings, or any number of data formats such as ints and floats. Each tag has a specific format that contains several parts:

  1.      Offset:          The data stream sample that the tag is associated with
  2.      Key:              A PMT symbol that contains an identifier for the tag
  3.      Value:           A PMT representation of the data the tag is passing
  4.      Source ID:    A PMT symbol representing the block that created the tag (optional)

Saving Tags To File

In my last post I went through using file sinks to process data offline using software such as MATLAB. Currently, the file sinks don’t save tags along with that data. However, such information such as the timing and value of tags might be useful for debugging. For example, if you have a tag denoting the start of each packet you may wish to have that timing information available in MATLAB.

To do this I wrote a fairly simple block “Tag to Byte”. For a parameter it asks for the key associated with a given tag, which in this example is “packet_len”. It can be inserted anywhere into a stream, just change the IO Type parameter to match the input data type. The Tag to Byte block can be followed by a file sink.

Tag To Byte

 

Tag to Byte Block In A Flowgraph

 

The block searches the stream for a tag matching the key. When it finds one it outputs a ‘1’ for that sample only. For any samples that don’t have a tag it outputs a ‘0’. This way the timing between tags is preserved as it is written to file. The file length will be the same as a file containing the data from the same point in the flow graph and the tags will be at the correct offset. The extra timing information from the tags can be used in processing data offline.

Tag and Data

 

Plotting Data and Tags in MATLAB

 

Adding Tags From A File

There also is some utility in being able to add tags anywhere in a data stream. GNU Radio has some functionality for this already. There is a stream to tagged stream block that will add tags at evenly spaced intervals. This can be useful for data that is evenly spaced. However if the tags need to correspond to data that is unevenly spaced, such as simulated irregular bursty transmissions we need another tool to do so.

A block I’ve written called “Add Tag at Offset” takes care of this. It can be inserted anywhere in a data stream and will insert tags into the stream. As parameters it will ask for the key to be associated with the tags and the value to insert. The last parameter is the offset for the tags, and this can be a vector of any length.

Add Tag at Offset

Using Add Tag at Offset In A Flow Graph

 

Every time the block reaches a sample with the same offset as is in its offset vector it will add a tag to that sample. All samples are passed through so the block does not interrupt the data flow.

This can be useful if we wish to take a block that uses tags in a complex flow graph and examine its performance by itself. We can load data from a file and add tags anywhere in the input stream. From the block’s perspective it seems as if it is still in the overall flow graph.

Both of these blocks and a few more that are useful for working with stream tags are available on GitHub.

What’s Next

There are a few obvious improvements to make to these tools that will be coming soon. It would be nice if the Add Tag at Offset block was able to add tags of different values instead of writing the same value each time. Its current operation might be okay if you were say passing the length of packets to the next block but less useful if you were using the tag to carry information that changes from packet to packet (an estimate of the packet’s carrier frequency offset, for example). A future improvement would be to have this block read the tag values from a vector as well.

It would also be nice if Tag to Byte would pass the value of the tags along to a file sink instead of just a ‘1’ to represent the presence of a tag. A new block with the working title “Tag Extractor” is being developed that will do just that. It will take some doing to cover all stream data types and tag data types, but I’m working on it. The first version of this (only for complex streams and complex-valued tags) is also available on GitHub.

 

GNU Radio: Tools for Offline Processing With MATLAB

Some Background

The GNU Radio project is pretty awesome.

My research focusses on wireless communications. The concept of radio receiver incredibly simple: use an antenna to receive a signal through the air and process that signal to extract the useful information (voice, data, etc). A transmitter is the same idea, in reverse order. The challenging part is the signal processing turn radio waves into information you care about.

When doing work in signal processing for radio communications it is very valuable to be able to test out your ideas with real radios transmitting over the air. In the not-to-distant past this would mean fabricating a new circuit board for every new design. The concept behind software-defined radio is to replace the signal processing done by the fixed hardware with signal processing done digitally. With a software-defined radio we can change the radio just by changing code and test out many new radio designs without having to build new hardware each time.

GNU Radio provides an excellent framework for experimenting signal processing for communication. A receiver or transmitter design in GNU Radio is made up of many blocks each performing one specific function (modulation, encoding, scaling, etc). Each block, written in C++ or Python, takes in data through an input port, processes it, and outputs it to an output port. GNU Radio provides the framework to glue the inputs and outputs together to form a chain of blocks to accomplish a task. This collection of connected blocks is called a flow graph. It even comes with a Simulink-style GUI where connections are shown graphically.

An Example Flowgraph What A Flow Graph in GNU Radio Companion Looks Like

 

With a flow graph implementing the signal processing necessary for, let’s say, an FM radio receiver, you can connect a software-defined radio to your PC and process the data from raw signal to recovered audio.

GNU Radio also gives users the ability to save data at any point in the flow graph to a binary file using file sinks and file sources. The resulting files can be read after the flow graph has run and are useful tools for debugging complicated flow graphs.

Connecting A File Sink Connecting A File Sink

 

Working With File Sinks

The file sink block uses stdio to write raw samples to a file. By default, during this process a certain number of samples will be buffered before being written to file. The number of samples varies by operating system. If there is only a short amount of data going into that file sink you may find the file to be empty after the flow graph is stopped. For this case, we can use the “unbuffered” option. This bypasses the buffering operation and writes all samples directly to file.

Another thing to note is that GNU Radio companion uses absolute file paths. So be sure to change the paths if you move your flow graph to another directory or machine.

There are several ways that data can be represented in GNU Radio. These data types can be selected in the file sink block and must match the data type of the samples that are streaming to it. The data types used are:

Complex 32 bit floating point for both I and 32 Q
Float 32 bit single precision floating point
Integer 32 bit signed integer
Short 16 bit signed integer
Byte 8 bit signed integer

 

Reading Binary Files in MATLAB

Once data has been written to a file there are several ways we can examine it. A file source can be used to read back the file into another GNU Radio flow graph. We could also use Python to read and analyze the data. While it’s not free, MATLAB can be a particularly useful tool and one which most engineers are rather familiar with.

Raw data files can be read into MATLAB quite easily with a few lines of code:

f = fopen(filename, 'rb');
v = fread(f,count);
fclose(f);

The first line tells MATLAB to open the file given by filename as read-only with Big-Endian encoding. The file identifier is stored in the variable f. The last line closes the file.

The MATALB function fread does the work of pulling data out of the file and into the array v. We give it the file identifier and the number of items to read (or tell it to read the file until empty). By default, MATLAB will read the file assuming the data type is an 8-bit integer. To read different file types we will need to add an extra parameter. For example, to read floats from a file we will use

v = fread (f, count, 'float');

For the case of complex numbers the I and Q samples are interleaved. The data file will contain the I for sample 1, the Q for sample 1, the I for sample 2, etc. The read_complex_binary.m file distributed with GNU Radio shows how to take care of this

t = fread (f, [2, count], 'float');
v = t(1,:) + t(2,:)*i;
[r, c] = size (v);
v = reshape (v, c, r);

We will read the data two floats at a time. Then convert the two floats to one complex number. That will then be reshaped into one complex vector.

It is convenient to create a function that does the file read operation for each data type. Several of these files come with GNU Radio such as read_float_binary.m and read_complex_binary.m. I’ve written functions that do the same for all the other data types used in GNU Radio. They are available to download from GitHub.

Writing Binary Files From MATLAB

We can also write data from MATLAB to a file and play that file back in GNU Radio using a file source. We’ll use a similar set of commands.

f = fopen (filename, 'wb');
v = fwrite (f, data);
fclose (f);

We’ll open a file specified by filename, this time making it writable. We’ll write the vector data into that file using MATLAB’s fwrite command. The variable v will show the number of elements in data that were successfully written to file.

We will need to give fwrite the same parameters to deal with different data types as in fread. For example, to write a vector of floats to file we will use

v = fwrite (f, data, 'float');

As with the read functions, there are several MATLAB files distributed with GNU Radio to handle writing to files from MATLAB. I’ve added functions to handle other data types which are also available in the GitHub repository.

 

Building A Home Lab: The Power Supply

The Motivation

Electronics labs at our universities and workplaces can have good sides and bad sides. Mine, for example, is filled with great equipment and knowledgeable people. However, it has a sum total of zero windows to the outside world. Occasionally it is nice to work on side projects at home at your own leisure. As a graduate student, funds for personal electronic equipment aren’t always plentiful so I’m trying to outfit myself with as much homemade equipment as possible. This is the start of what I’m hoping will be several posts about that process. I’ll be starting with one of the most basic pieces of equipment an EE needs, a power supply.

The Supplies

We’re going to be starting with a desktop computer power supply and convert it into something more friendly for electronics work. They look something like this:

Supply With Lid Off

Mine was salvaged out of one of those black Dell computers that universities like to buy (and later throw away) by the dozens. Inside they have a fan, transformer, regulator and some accompanying circuitry, and a rainbow of wires leading to some Molex connectors.

Warning: The power supply contains two very large capacitors that can hold a charge for long after the unit is unplugged. Let the power supply sit unplugged for several days before trying to open it up.

We’ll be wiring in 5 binding posts, 4 for our regulated voltages and 1 for ground. Sparkfun sells individuals for a few cents. I made each of the voltages red and the ground black. You can also choose different colors for each of the regulated voltages.

I also used an SPST switch I had laying around (I’m pretty sure I bought it from RadioShack at one time) as well as an LED and accompanying resistor as an indicator. You’ll also need a 10Ω 10-Watt power resistor to ensure our supply operates correctly.

I also gathered some other supplies for the build:

Build Supplies

  • Soldering Iron and Solder
  • Solder Helping Hands
  • Multimeter
  • Superglue
  • Heat Shrink Tubing
  • A Good Beer (not required, but highly recommended)

The Build

The first order of business is mounting the binding posts. I drilled five holes in the top of the power supply case. The posts pop right in and are held in place with a washer and nut. Next, a hole for the switch is drilled in the front of the case and the switch is mounted. Be sure to position the switch such that the case fits back on the power supply without bumping into anything.

Case With Hardware

Inside the power supply there will be many different colored wires. Most of these power supplies have a standard color scheme, but make sure to check with a multimeter to make sure yours follows it:

  • Orange: 3.3V
  • Red: 5V
  • Yellow: 12V
  • Blue: -12V
  • Black: Ground

In addition there will be three additional wires:

  • Green: Turns on the supply when shorted to ground
  • Purple: Puts out +5V when the supply is plugged in and off
  • Gray: Puts out +5V when supply is plugged in and on

We’ll connect one terminal of an SPST switch to Green and the other terminal to ground. When the switch is closed the power supply should turn on. The purple wire won’t be used, but could possibly drive a second LED to let us know that our supply is plugged in but the switch is off. The gray wire will be connected to our status LED that shows the supply is on and running. Connect the gray wire to a resistor. Connect the resistor to the anode of an LED and connect the cathode to ground. The value of the resistor can be sized depending on the LED voltage drop and current maximum. I used a 330Ω resistor which might have been overkill for a blue LED with such a high voltage drop, but we don’t need the LED to be very bright.

There’s one more thing to add to the case. Most of these power supplies expect a certain minimum load while they’re operating. If there’s no load on the supply sometimes the ±12V rails won’t be regulated to the correct voltage. In order for the power supply to behave properly we’ll connect a 10Ω resistor to the 5V rail to give the supply a constant load. Since there will be quite a bit of power dissipated through this guy, I chose a 10-Watt sandbar resistor. The resistor is superglued to the side of the case and connected between 5V and ground.

External Components Diagram

 

The step is fairly straightforward. Solder the four regulated voltages and the ground to the binding posts we attached to the case. We only need one wire per voltage so we can get rid of the rest to save some room inside the case. We could desolder them all from the board but I found that cutting them off was much easier. If there is any exposed metal where the cuts have been made we can cover the ends with a small bit of heat shrink to prevent any accidental shorting.

External Components WiringBefore plugging the supply in check that all the connections have been made correctly. Wrap all connections between wires or use heat shrink to make sure nothing is exposed. When the device is first plugged in all the posts should be at 0V. When the switch is turned ON the LED should light and each post should read the correct voltage. Each post should be labeled. I used a few zip ties to tame the remaining wires and put the case back on the unit. The wires should fit nicely in the space in the front of the unit.

Finished Case