Using Sparse tools for my internship project

What is Sparse ?

Sparse or semantic Parser for C  is a tool built by Linus Torvald and colleagues with the aim of enforcing type-checking.  It is actually a static analysis tool. By static we mean it analyses code without executing it; Sparse converts a sequence of characters in the program into a sequence of tokens and analyses the C syntax used.

By token, we mean strings with an assigned or identified meaning. These can be for example keywords, an identifier, a constant, a string, a special symbol, or an operator. A token can also be seen as the smallest element of a program that is meaningful to the compiler.

Sparse is all about semantics (tokens or groupings of tokens) or context(whether a group of tokens belong to a release or acquires or holds or no group).

The semantics parsing process in Sparse involves figuring out which grouping the tokens are and what _types_ are this grouping implies.

Sparse can work as a front end of a compiler and figures out whether a grouping of tokens belongs to an acquire or release lock for example.

The parsing in Sparse is theoretically done in five phases:

- full-file tokenization

- pre-processing

- semantic parsing

- lazy type evaluation

- inline function expansion and tree simplification

The above five phases are well described in the link below

link to sparse details.

The tool is currently maintained by Luc Van Oostenryck.

Why Sparse?

There are many tools,  out there in the wild, that Kernel developers and newbies use to find and fix problems in a source code. Each tool excels in its own way.

Typical examples are checkpatch; a coding style checker,  Coverity; a commercial tool that fixes Linux code, Coccinelle; used for Linux code matching and transformation, and smatch, to mention but a few.

Why Sparse? Sparse summarises particular features of some codes, find particular possible troublesome patterns and makes it easier to understand or navigate around a large code set. The tool is structured as a library that other tools can easily incorporate. (eg make C=2)

If you are working on a project that requires fixing lock warning as I am or address warning. Sparse will be the most appropriate to find these types of warnings or errors.

Install Sparse

The latest released versions can be downloaded from the Sparse homepage at

https://sparse.wiki.kernel.org/index.php/Main_Page

Alternatively, you can get snapshots of the latest development version of sparse using git to clone:

git clone git://git.kernel.org/pub/scm/devel/sparse/sparse.git

DaveJ has hourly generated tarballs of the git tree available at:

http://www.codemonkey.org.uk/projects/git-snapshots/sparse/

Once you have it, just do:

make
make install

as a regular user, and it will install sparse in your ~/bin directory.

Distributions like Fedora or Ubuntu made things easier, a simple command on a terminal will install Sparse.

sudo dnf install sparse -y

sudo apt install sparse -y

How to use Sparse?

The tool was designed to be particularly small, simple, and easy to use.

Run Sparse on Kernel C files that would get re-compiled only saving our time if in a hurry:

make C=1 pwd_directory/

Run Sparse on all kernel C files regardless of whether they need to get re-compiled or not:

make C=2 pwd_directory/

pwd_directory/ can be any kernel directory where our target C source file(s) is (are) located.

Examples: make C=2 drivers/staging/; make C=1 drivers/staging/

When running make to detect possible coding faults, Sparse offers flexibility in which type of warning or errors to be displayed.  For that, a variable CF can be used to pass arguments to sparse.

The build system passes -Wbitwise to sparse automatically.

There are many options : -Wsparse-all, -Waddress-space, -Wcast-to-as, -Wcast-truncate, etc.

Sparse can also be used as a stand-alone application;

if that is the case, just type the command

sparse $filename

$filename for any C files.

Sparse provides a set of annotations designed to convey semantic information about types, such as what address space pointers point to, or what locks a function acquires or releases.

Advertisement

Experience applying to Outreachy Internship program

First encounter

I first learned about  Outreachy  at  The Open Source Summit 2018 in Edinburgh, Scotland.

I attended a Linux Kernel security talk where Julia Lawall, a research scientist at INRIA presented her work on Coccinelle – work which I found interesting. At the end of her talk, she briefly spoke about an internship that she helps run, Outreachy,  a scheme that was originally developed to help women step into open source technology but has now started considering also other minorities and people with some circumstances, like  someone who wants to change his/her career.

I became interested in applying  after two interns stepped forward and presented their projects. I of course interacted with them  and Julia after the talk.

Internship application experience

Before being successful, I applied for the Outreachy internship three times;

The first time was in December 2018,  I started the application when the deadline was almost passed. Though I found all the project quite interesting, I did not have a clear picture of which project I really wanted to do so I applied to multiple ones. I was not successful.  After self-analysis, I realised I did not meet the minimum requirements which is 10 patches and two series accepted. I was not very sad because I realised I just learned the basics of git the hard way. This was my first time hands on git and the Kernel.

The second time I did not dedicate much time to the application process. The third time I knew exactly which project fits my profile. I went for them and ensured the minimum requirement is met.  The two projects I applied for were really fascinating for me. They both had  mentors who were willing to help me understand what they want. I told myself I will try to research on them. I had already read about C programming  and  some kernel programming it made it easy for me to submit patches as I had an understanding of what was done when I saw the code.

I sometimes sent patches  that ended up not being accepted. I tried to learn from the failed patches and  re-submitted after  improvement and they normally got accepted after a slight change.  The feedback received from the sent ones was really important.

Being a newbie is fun, I enjoy the application process especially the interaction I had with the experts and the guidance of Greg and Julia Lawall. I made a lot of mistakes and learned from them. I also interacted with one of the other applicant asking her some advice.

Advice

If you are planning to apply for the Outreachy internship, my advice to you is apply, do not hesitate, pick one or two projects that meet your passion  (there  are a range of projects that require different skills from just writing to programming skills) and also check the likelihood of you getting the project. If it happens that you do not get accepted, do not take it personally, it just means you did not qualify yet, try to self analyse, improve on your skills and try again until you qualify next time 🙂