Writing a Linux device driver module for kernels 2.6 or later with udev

This is a short tutorial for a sample character device module aimed at Linux kernels 2.6 and later (including 3.x and 4.x) using udev.
I thought might as well produce my own tutorial, after finding that most of the ones floating around the net are either lacking in terms of features, or not as up to date as could be with regards to the latest kernel advancements.

Some of the features demonstrated with this sample are:

• creation of a character device as a module
• use of udev, for the device instantiation
• use of a kernel FIFO, for data queuing
• use of a mutex, to prevent concurrent access to the same device
• use of sysfs, as a means to add data to the queue
• detection of access mode on open, to ensure read-only access
• provision of module options during insmod, to enable debug output

Not bad for a short sample, eh?

Of course, this example is just there to help get you started, and does not replace a proper guide to Linux driver development, such as the freely available and strongly recommended "Linux Device Drivers - Third Edition" from O'Reilly.


What's the deal with udev?

If you have some basic knowledge of device drivers on UNIX systems, you probably are already aware that devices are instantiated into the dev/ directory through the use of a (major, minor) pair of integers. UNIX veterans will no doubt recall how they used to call on mknod to create their devices, and some UNIX systems still require that. Well into the 2.6 kernels, Linux was also more or less using the statically allocated major/minor approach, through its devfs implementation. At one stage however, kernel developers got slightly annoyed about having to coordinate the allocation of major/minor numbers, not being able to keep /dev as clean and as abstracted as they liked and also feared that Linux would run out of numbers to assign. Because of that, they developed a new approach to device instantiation called udev, which led them to eventually ditch devfs altogether in kernel 2.6.18. For more information about udev vs devfs, you may want to read Greg Kroah-Hartman's The final word from December 2003.

How this matters to you as a driver writer is that a lot of existing tutorials you will find rely on devfs or static allocation of a major and minor number, which doesn't apply to recent kernels. Thus, you want a tutorial that demonstrates how to use udev, such as the one provided here.
For the record, this sample driver was tested with kernel 3.0.3 as well as 2.6.39.


Device description

The origin of this device driver sample mostly stems from wanting to craft a virtual device to simulate an HDMI-CEC enabled TV, in order to facilitate the development of libcec/cecd. Basically, with such a driver, a libcec application can connect to a virtual device that plays a pre-recorder set of data, which avoids the need to have an actual HDMI-CEC device plugged in or even a development platform that has an HDMI port.

Since the driver above is fairly straightforward to write, it is a good candidate to serve as a base for a sample. In order to turn it into something that is generic enough for a tutorial, we did remove anything that was HDMI-CEC related and turned the driver into one that can store sequences of character data and then repeat them in the order they were received, pretty much as a FIFO for text string. Given this feature description, it makes a lot of sense to call our little device 'parrot'.

For this 'parrot' device then, what we need first is a buffer to store data. The smart approach here, rather than reinvent the wheel is to use a Kernel FIFO, as this is a feature provided by the kernel and specifically aimed at device drivers. This will greatly simplify our design and of course, if you use this sample as a base for a driver that does actual hardware I/O, knowing how to use a Kernel FIFO will no doubt come handy.

Now, while we obviously are going to use read accesses to the actual device (cat /dev/parrot) to read FIFO messages, we are not going to use write accesses to the device to fill the FIFO. Instead we will use sysfs, and make our device read-only.
This may sound counter-intuitive, aside from wanting to demonstrate the use of sysfs, but the reason is, in a typical virtual repeater device scenario, you would use separate FIFOs for read and write, or a FIFO for read and a sink for write, since you don't want data written by your application to the device to interfere the pre-defined scenario you have for readout. Of course, if you want to add a device write endpoint to our sample device, it is a simple matter of copy/pasting the read procedures, and modify them for write access.

Finally, because this is a sample, we will add some debug output which we will toggle using a module parameter on insmod as it is a good example to demonstrate how driver load options can be handled.


Source Walkthrough

The open()/close() calls should be fairly straightforward. The open() call demonstrates how one can use the struct file f_flags attribute to find out whether the caller is trying to use the device for read-only, write-only or read/write access. As indicated, we forcibly prevent write access, and return an error if the device is open write-only or read/write. Then we use the non-blocking mutex_trylock() and mutex_unlock(), along with a global device mutex, to prevent more than one process from accessing the device at any one time. This way, we won't have to worry about concurrent read accesses to our Kernel FIFO.

The device_read() call uses the kfifo_to_user() to copy data from the Kernel FIFO into userspace. One thing you must be mindful of, when writing device drivers, is that kernelspace and userspace are segregated so you can't just pass a kernel buffer pointer to a user application and expect it to work. Data must be duplicated to or from userspace. Hence the _to_user call. Now, because we store multiple messages in a flat FIFO, and don't use specific message terminators such as NUL, we need a table to tell us the length of each message. This is what the parrot_msg_len[] (circular) table provides. For this to work, we also need 2 indexes into it: one to point to the length of the next message to be read (parrot_msg_idx_rd), and one to point to the next available length entry for a write operation (parrot_msg_idx_wr). We could also have used _enqueue and _dequeue for the suffixes, as this is what you will generally find in a driver source.

The one_shot and message_read variables are used to alleviate a problem you will face if you use cat to read from the device. The problem with cat is that it will continue to issue read requests, until the device indicates that there is no more data to be read (with a read size of zero). Thus, if left unchecked, cat will deplete the whole FIFO, rather than return a single message. To avoid that, and because we know that cat will issue an open() call before attempting to read the device, we use the boolean message_read to tell us if this is our first read request since open or not. If it isn't the first request, and the one_shot module parameter is enabled (which is the default), we just return zero for all subsequent read attempts. This way, each time you issue a cat, you will at most read one message only.

With our open(), close() and read() calls properly defined, we can set the file operations structure up, which we do in the static struct file_operations fops. For more on the file operations which you can use for character devices, please see Chapter 3 of the Linux Device Drivers guide.

All the above takes care of our basic driver workhorse. Yet we still haven't provided any means for the user to populate the FIFO. This is done through the next subroutine called sys_add_to_fifo().

The first thing we do there is check for overflows on either the FIFO or the message length table. Then, we use kfifo_in() to copy the data provided by the user to the sysfs endpoint (see below). Note that, in this case, the sysfs mechanisms have already copied the data provided by the user into kernelspace, therefore using kfifo_in() just works, and there is not need to call kfifo_from_user(), as we would have had to do if we were writing a regular driver write procedure.

The other sysfs endpoint we provide is a FIFO reset facility, implemented in sys_reset, which is very straightforward.

With our sysfs functions defined, we can use the DEVICE_ATTR() macros to define the structure which we'll have to pass to the system during the module initialization. The S_IWUSR parameter indicates that we want users to have write access to these endpoints.

Now, we come to the last part of our driver implementation, with the actual module initialization and destruction. In module_init() the first thing we do, and this is the part where we rely on udev, is to dynamically allocate a major, through register_chrdev(). This call takes 3 parameters: The first one is either a specific major number, or 0 if you want the system to allocate one for you (our case), the second is a string identifier (which can be different from the name you will use in /dev) and the last parameter is a pointer to a file operations structure.

Next, we have a choice of creating a device associated with a class or a bus. For a simple sample such as ours, and given that our device is not tied to any hardware or protocol, creating a bus would be an overkill, so we will go with a class. For laymen, the difference between using a bus and a class is that you will get your devices listed in /sys/devices/<classname> in sysfs, instead of /sys/devices/virtual, and you will also have them under <classname> in /dev (though it is possible to achieve the same if you add a "/" in your device name when calling device_create()).

With a class established, we can then call device_create() which performs the actual instantiation of our device. The description of the device_create() parameters can be found here. We use the MKDEV() macro to create a dev_t out of the major we previously obtained, using 0 as the minor as we only have one device. It is at this stage that a "parrot_device" gets created in /dev, using the parameter provided for the name.

The next calls to device_create_file() are used to create the two sysfs endpoints, fifo and reset, out of the structures previously defined with DEVICE_ATTR(). Because we use a class, this results in the /sys/devices/virtual/parrot/parrot_device/fifo and /sys/devices/virtual/parrot/parrot_device/reset endpoints being created.

Finally we end the module_init() call with the remainder of the device initialization (FIFO, mutex, indexes).

The remainder of the code should be fairly explicit so I won't comment on it.


Source:

parrot_driver.h:

/*
 * Linux 2.6 and later 'parrot' sample device driver
 *
 * Copyright (c) 2011-2015, Pete Batard <pete@akeo.ie>
 *
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program. If not, see <http://www.gnu.org/licenses/>.
 */

#define DEVICE_NAME "device"
#define CLASS_NAME "parrot"
#define PARROT_MSG_FIFO_SIZE 1024
#define PARROT_MSG_FIFO_MAX  128

#define AUTHOR "Pete Batard <pete@akeo.ie>"
#define DESCRIPTION "'parrot' sample device driver"
#define VERSION "1.2"

/* We'll use our own macros for printk */
#define dbg(format, arg...) do { if (debug) pr_info(CLASS_NAME ": %s: " format, __FUNCTION__, ## arg); } while (0)
#define err(format, arg...) pr_err(CLASS_NAME ": " format, ## arg)
#define info(format, arg...) pr_info(CLASS_NAME ": " format, ## arg)
#define warn(format, arg...) pr_warn(CLASS_NAME ": " format, ## arg)

parrot_driver.c:

/*
 * Linux 2.6 and later 'parrot' sample device driver
 *
 * Copyright (c) 2011-2015, Pete Batard <pete@akeo.ie>
 *
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program. If not, see <http://www.gnu.org/licenses/>.
 */

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/device.h>
#include <linux/types.h>
#include <linux/mutex.h>
#include <linux/kfifo.h>
#include "parrot_driver.h"

/* Module information */
MODULE_AUTHOR(AUTHOR);
MODULE_DESCRIPTION(DESCRIPTION);
MODULE_VERSION(VERSION);
MODULE_LICENSE("GPL");

/* Device variables */
static struct class* parrot_class = NULL;
static struct device* parrot_device = NULL;
static int parrot_major;
/* Flag used with the one_shot mode */
static bool message_read;
/* A mutex will ensure that only one process accesses our device */
static DEFINE_MUTEX(parrot_device_mutex);
/* Use a Kernel FIFO for read operations */
static DECLARE_KFIFO(parrot_msg_fifo, char, PARROT_MSG_FIFO_SIZE);
/* This table keeps track of each message length in the FIFO */
static unsigned int parrot_msg_len[PARROT_MSG_FIFO_MAX];
/* Read and write index for the table above */
static int parrot_msg_idx_rd, parrot_msg_idx_wr;

/* Module parameters that can be provided on insmod */
static bool debug = false; /* print extra debug info */
module_param(debug, bool, S_IRUGO | S_IWUSR);
MODULE_PARM_DESC(debug, "enable debug info (default: false)");
static bool one_shot = true; /* only read a single message after open() */
module_param(one_shot, bool, S_IRUGO | S_IWUSR);
MODULE_PARM_DESC(one_shot, "disable the readout of multiple messages at once (default: true)");


static int parrot_device_open(struct inode* inode, struct file* filp)
{
 dbg("");

 /* Our sample device does not allow write access */
 if ( ((filp->f_flags & O_ACCMODE) == O_WRONLY)
   || ((filp->f_flags & O_ACCMODE) == O_RDWR) ) {
  warn("write access is prohibited\n");
  return -EACCES;
 }

 /* Ensure that only one process has access to our device at any one time
  * For more info on concurrent accesses, see http://lwn.net/images/pdf/LDD3/ch05.pdf */
 if (!mutex_trylock(&parrot_device_mutex)) {
  warn("another process is accessing the device\n");
  return -EBUSY;
 }

 message_read = false;
 return 0;
}

static int parrot_device_close(struct inode* inode, struct file* filp)
{
 dbg("");
 mutex_unlock(&parrot_device_mutex);
 return 0;
}

static ssize_t parrot_device_read(struct file* filp, char __user *buffer, size_t length, loff_t* offset)
{
 int retval;
 unsigned int copied;

 /* The default from 'cat' is to issue multiple reads until the FIFO is depleted
  * one_shot avoids that */
 if (one_shot && message_read) return 0;
 dbg("");

 if (kfifo_is_empty(&parrot_msg_fifo)) {
  dbg("no message in fifo\n");
  return 0;
 }

 retval = kfifo_to_user(&parrot_msg_fifo, buffer, parrot_msg_len[parrot_msg_idx_rd], &copied);
 /* Ignore short reads (but warn about them) */
 if (parrot_msg_len[parrot_msg_idx_rd] != copied) {
  warn("short read detected\n");
 }
 /* loop into the message length table */
 parrot_msg_idx_rd = (parrot_msg_idx_rd+1)%PARROT_MSG_FIFO_MAX;
 message_read = true;

 return retval ? retval : copied;
}

/* The file_operation scructure tells the kernel which device operations are handled.
 * For a list of available file operations, see http://lwn.net/images/pdf/LDD3/ch03.pdf */
static struct file_operations fops = {
 .read = parrot_device_read,
 .open = parrot_device_open,
 .release = parrot_device_close
};

/* Placing data into the read FIFO is done through sysfs */
static ssize_t sys_add_to_fifo(struct device* dev, struct device_attribute* attr, const char* buf, size_t count)
{
 unsigned int copied;

 dbg("");
 if (kfifo_avail(&parrot_msg_fifo) < count) {
  warn("not enough space left on fifo\n");
  return -ENOSPC;
 }
 if ((parrot_msg_idx_wr+1)%PARROT_MSG_FIFO_MAX == parrot_msg_idx_rd) {
  /* We've looped into our message length table */
  warn("message length table is full\n");
  return -ENOSPC;
 }

 /* The buffer is already in kernel space, so no need for ..._from_user() */
 copied = kfifo_in(&parrot_msg_fifo, buf, count);
 parrot_msg_len[parrot_msg_idx_wr] = copied;
 if (copied != count) {
  warn("short write detected\n");
 }
 parrot_msg_idx_wr = (parrot_msg_idx_wr+1)%PARROT_MSG_FIFO_MAX;

 return copied;
}

/* This sysfs entry resets the FIFO */
static ssize_t sys_reset(struct device* dev, struct device_attribute* attr, const char* buf, size_t count)
{
 dbg("");

 /* Ideally, we would have a mutex around the FIFO, to ensure that we don't reset while in use.
  * To keep this sample simple, and because this is a sysfs operation, we don't do that */
 kfifo_reset(&parrot_msg_fifo);
 parrot_msg_idx_rd = parrot_msg_idx_wr = 0;

 return count;
}

/* Declare the sysfs entries. The macros create instances of dev_attr_fifo and dev_attr_reset */
static DEVICE_ATTR(fifo, S_IWUSR, NULL, sys_add_to_fifo);
static DEVICE_ATTR(reset, S_IWUSR, NULL, sys_reset);

/* Module initialization and release */
static int __init parrot_module_init(void)
{
 int retval;
 dbg("");

 /* First, see if we can dynamically allocate a major for our device */
 parrot_major = register_chrdev(0, DEVICE_NAME, &fops);
 if (parrot_major < 0) {
  err("failed to register device: error %d\n", parrot_major);
  retval = parrot_major;
  goto failed_chrdevreg;
 }

 /* We can either tie our device to a bus (existing, or one that we create)
  * or use a "virtual" device class. For this example, we choose the latter */
 parrot_class = class_create(THIS_MODULE, CLASS_NAME);
 if (IS_ERR(parrot_class)) {
  err("failed to register device class '%s'\n", CLASS_NAME);
  retval = PTR_ERR(parrot_class);
  goto failed_classreg;
 }

 /* With a class, the easiest way to instantiate a device is to call device_create() */
 parrot_device = device_create(parrot_class, NULL, MKDEV(parrot_major, 0), NULL, CLASS_NAME "_" DEVICE_NAME);
 if (IS_ERR(parrot_device)) {
  err("failed to create device '%s_%s'\n", CLASS_NAME, DEVICE_NAME);
  retval = PTR_ERR(parrot_device);
  goto failed_devreg;
 }

 /* Now we can create the sysfs endpoints (don't care about errors).
  * dev_attr_fifo and dev_attr_reset come from the DEVICE_ATTR(...) earlier */
 retval = device_create_file(parrot_device, &dev_attr_fifo);
 if (retval < 0) {
  warn("failed to create write /sys endpoint - continuing without\n");
 }
 retval = device_create_file(parrot_device, &dev_attr_reset);
 if (retval < 0) {
  warn("failed to create reset /sys endpoint - continuing without\n");
 }

 mutex_init(&parrot_device_mutex);
 /* This device uses a Kernel FIFO for its read operation */
 INIT_KFIFO(parrot_msg_fifo);
 parrot_msg_idx_rd = parrot_msg_idx_wr = 0;

 return 0;

failed_devreg:
 class_destroy(parrot_class);
failed_classreg:
 unregister_chrdev(parrot_major, DEVICE_NAME);
failed_chrdevreg:
 return -1;
}

static void __exit parrot_module_exit(void)
{
 dbg("");
 device_remove_file(parrot_device, &dev_attr_fifo);
 device_remove_file(parrot_device, &dev_attr_reset);
 device_destroy(parrot_class, MKDEV(parrot_major, 0));
 class_destroy(parrot_class);
 unregister_chrdev(parrot_major, DEVICE_NAME);
}

/* Let the kernel know the calls for module init and exit */
module_init(parrot_module_init);
module_exit(parrot_module_exit);

The relevant Makefile is provided in the parrot source archive (link below).

Testing:

root@linux:/ext/src/parrot# make
make -C /usr/src/linux SUBDIRS=/mnt/hd/src/parrot modules
make[1]: Entering directory `/mnt/hd/src/linux-3.0.3'
  CC [M]  /mnt/hd/src/parrot/parrot_driver.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /mnt/hd/src/parrot/parrot_driver.mod.o
  LD [M]  /mnt/hd/src/parrot/parrot_driver.ko
make[1]: Leaving directory `/mnt/hd/src/linux-3.0.3'
root@linux:/ext/src/parrot# insmod parrot_driver.ko debug=1
root@linux:/ext/src/parrot# lsmod
Module                  Size  Used by
parrot_driver           4393  0
root@linux:/ext/src/parrot# ls -lF /dev/parrot_device
crw------- 1 root root 251, 0 2011-08-22 22:04 /dev/parrot_device
root@linux:/ext/src/parrot# ls -lF /sys/devices/virtual/parrot/parrot_device/
total 0
-r--r--r-- 1 root root 4096 2011-08-22 22:05 dev
--w------- 1 root root 4096 2011-08-22 22:05 fifo
drwxr-xr-x 2 root root    0 2011-08-22 22:05 power/
--w------- 1 root root 4096 2011-08-22 22:05 reset
lrwxrwxrwx 1 root root    0 2011-08-22 22:04 subsystem -> ../../../../class/parrot/
-rw-r--r-- 1 root root 4096 2011-08-22 22:04 uevent
root@linux:/ext/src/parrot# echo "this should generate an error" > /dev/parrot_device
-bash: /dev/parrot_device: Permission denied
root@linux:/ext/src/parrot# echo "Yabba Dabba Doo" > /sys/devices/virtual/parrot/parrot_device/fifo
root@linux:/ext/src/parrot# echo "Yabba Dabba Daa" > /sys/devices/virtual/parrot/parrot_device/fifo
root@linux:/ext/src/parrot# cat /dev/parrot_device
Yabba Dabba Doo
root@linux:/ext/src/parrot# cat /dev/parrot_device
Yabba Dabba Daa
root@linux:/ext/src/parrot# echo "test" > /sys/devices/virtual/parrot/parrot_device/fifo
root@linux:/ext/src/parrot# echo "test" > /sys/devices/virtual/parrot/parrot_device/fifo
root@linux:/ext/src/parrot# echo 1 > /sys/devices/virtual/parrot/parrot_device/reset
root@linux:/ext/src/parrot# cat /dev/parrot_device
root@linux:/ext/src/parrot#

Links:

• The github repository, from which you can clone the source
• The latest 'parrot' sample Linux driver source, as a tar archive
• The complete PDF version of O'Reilly's Linux Device Drivers - Third Edition
• An introduction on the Linux 2.6 Device Model
• A primer on the kfifo implementation (See also /samples/kfifo/* in the Linux kernel source tree)