Robot framework supports several different formats for its test cases. You can write tests in HTML, reStructuredText, tab-separated values (eg: export from a spreadsheat program), and plain text with either pipe-separated fields or space-separated fields.

Each format has advantages and disadvantages, mostly centered around how easy it is to write the tests vs how easy it is to read the test after it has been created. These two goals are sometimes in conflict with each other.

This article gives a very brief rundown of the advantages of each, and covers why I eventually came to embrace the pipe-separated plain text format as the best of the available formats.

HTML

HTML is my least favorite format for test cases. This is due to two reasons: it can be difficult to write the tests, and it can be difficult comparing two versions of a test.

The tests are difficult to write because HTML editors aren’t optimized for writing robot, so you’re trying to write test cases in a tool designed to create websites. The tool has a different goal than the user. Of course, you can use a plain text editor, but then you’re cursed with having to hand-edit HTML tags (though, smart editors can help ease that pain).

Because your test case data is wrapped in HTML tags, it can make the test cases hard to diff and merge simply because of all of the extra markup. Worse, in my experience, different web editors may introduce differences in the markup by adding or removing whitespace or linebreaks. If the people in an organization use different tools, it can cause problems for comparing versions of a test case – a single line change to a test case in a different tool may end up reformatting the whole file.

reStructuredText

Using reStructuredText has an advantage similar to HTML in that you can create test documents that are very easy to read. Writing reStructured text is at least arguably easier to write than HTML since the markup doesn’t rely so much on tags as it does simple conventions, so you get the best of both words – fairly easy to write, and definitely easy to read.

What I don’t like about reStructuredText is that it’s still a bit verbose. While you can “draw” test tables that indeed look like tables, you have to draw test tables that look like tables. Since the focus of writing tests should be on the logic and data of the test, having to spend extra effort to make the tables look like tables is onerous.

There’s another option where you can use code blocks. Within a code block you can use the space-separated format. However, if you’re going to use the space-separated format, why add the extra complexity of reStructuredText on top of it?

I would be interested in hearing from someone that is successfully using reStructuredText for their tests. Maybe I can be convinced it’s the way to go.

Tab-separated values

My first exposure to robot was at a company where we settled on the tab-separated format. This is arguably the easiest to type when using a standard text editor, since it’s just a single keypress to start a new cell. The problem with tab-separated files, in my opinion, is the fact that it’s hard to distinguish between a tab and just a bunch of spaces. And, depending on your tab stops, a single tab may often look like a single space, leading to ambiguity.

You also have the same (dis?)advantage as using tabs in software – different people have different settings for tab stops. Some see this as an advantage, but often results in code that lines up and reads properly to one person but not another.

Plain text, space-separated format

The plain text, space-separated format seems to be the most popular. This is the format that the robot framework inventor uses, and many examples I see on the internet use this format. It has the advantage that test cases can read very much like English language, which definitely is one of the main appeals of using a keyword driven testing language.

The Achilles heal of this format, in my opinion, is that it leads to ambiguity. Two spaces can separate two cells in a row, but in print and on websites it’s can be difficult to visually distinguish one space from two. To work around that, the robot framework user guide suggest using four spaces, but then you lose some of the readability, plus it’s just a pain to type four spaces between each cell.

Plain text pipe-separated format

This is the format I prefer, because I think it strikes a nice balance between being easy to read and write. It has many of the same advantages as the space- and tab-separated formats, but it has one distinct advantage in that it is much less ambiguous. There’s no denying the presense of a pipe between two fields.

The downside to the pipe separated format is that pressing space-pipe-space is tedious, and also difficult to say out loud when teaching others. Another downside is that the presense of a pipe removes some of the “english-like” features of the space separated performance.

In fact, having to type a pipe is a pretty big downside. Just as I would not want to type four spaces between two cells, typing space-pipe-space is also somewhat annoying. Not only that, but on many non-US keyboards, the pipe character is particularly hard to type. If I had one of those keyboard I would definitely gravitate toward the space-separated format.

That, to me, is just about the only real downside to using the pipe-separated format. In fact, I liked the format enough, and hated typing the pipe enough, that I wrote my own editing tools to eliminate the pain.

A few years ago I wrote a desktop app that let you type a pipe simply by pressing the tab key. You then get the ease of editing that you would get with the tab-separated-value format, but with the clarity of the pipe-separated format.

This proved to be a fantastic solution. However, I was spending a lot of my time writing a basic text editor around this single feature, rather than focusing on adding even more features.

That’s when I switched to writing a robot extension for brackets. My extension has a couple of key features that I believe gives the best editing experience for robot framework tests. You have the ease of entering separators by pressing tab, the clarity that comes with pipes rather than spaces, yet the ease of reading because the pipes are rendered in a dim color, making them less dominant. And, because it is an extension to an existing editor, I can focus on adding additional features such as in-line keyword documentation, code hints, and so on.

Summary

I prefer the pipe-separated plain text format, and use that at work, and in my blog posts and answers on stackoverflow. It’s what I recommend to anyone just getting started. But honestly, all of the supported formats are great and have their place. I think that the fact they are all plain text gives robot a lot of its appeal, because the tests play very well with development tools we already use – text editors, source code control, diff, etc.