Oops

Sorry about planet.fd.o pollution...

I'll stop playing with the Blogger tags now...

m(_ _)m

Gallium3D: Introduction

At Tungsten Graphics I've been working in Gallium3D — a very promising architecture to develop 3D graphic drivers for multiple operating systems and graphic APIs.

It comes as no surprise that Gallium3D is a large and complex piece of software. So I've decided to write a bit about it in the hope of helping newcomers to get more quickly familiarized with it.

A few adverts: I had little to do with Gallium3D's design — that's the work of much brighter people such as Keith Whitwell and Brian Paul — so a lot of the rationale written here is partly my own speculation; also Gallium3D architecture is still under flux (much less than before, but still), so this refers to its current state.

Architecture

Gallium3D architecture can be described as a set of interfaces and a collection of supporting libraries.

Gallium3D is not a framework. All attempts of using the Hollywood Principle of "Don’t call us, we’ll call you." for 3D driver development imply making assumptions about the hardware behavior which quickly are proven wrong as new hardware generations come along. Instead, by simply providing a set of libraries, Gallium3D's can more easily adapt in this rapidly evolving field. So Gallium3D's principle is indeed "Is up to you to call us, as we won't call you". Is is necessary to have this principle in mind to understand how all Gallium pieces fit together.

Modules

State tracker

Translates the graphics API state (e.g., blend mode, texture sampling mode, etc.), shaders, and graphics primitives into something that the pipe driver below understands.

Pipe driver

Translates the state, shaders, and primitives in something that the hardware understands (e.g., register writes, shader/command buffers, and vertex buffers).

Winsys

Instantiates and binds all pieces above (state tracker and pipe driver) together with the OS, window system, and 2D display driver.

Auxiliary modules

Provide optional auxiliary services such as, software interpretation of shaders for hardware without hardware vertex shader support, state caching, buffer management, etc.

Module dependency table

	Graphics API	Graphics Hardware	OS
Auxiliary modules	No	No	No
State Tracker	Yes	No	Yes/No(1)
Pipe driver	No	Yes	No
Winsys	Yes	Yes	Yes

(1) The state tracker depends on the graphics API, so it can be made OS-independent for OS-independent APIs (such as OpenGL), but not for OS-dependent APIs (such as Direct3D)

The higher the module is in the previous table the more it is reused (auxiliary modules,State Tracker). The lower it is, more times it will have to be rewritten (Winsys). Although the dividing line between these modules is blurry, we are always interested in moving functionality upwards as much as possible. This is one of the areas where Gallium3D architecture is under flux: when we support a new graphics hardware, graphics API, or OS and realize that there is some functionality that can be generalized then we move it upwards; if we realize that previously made assumptions no longer hold, then we move that functionality downwards.

Interfaces

State tracker <-> Pipe driver

There is a per-context interface(p_context.h) and a global/per-screen interface(p_screen.h).

State tracker <-> Winsys

p_winsys.h

Pipe driver <-> Winsys

Besides p_winsys.h above, each pipe driver has its own additional winsys interface: sp_winsys.h (pure-software pipe driver), i915_winsys.h (Intel 915/945 pipe driver), etc.

Data flow

The data flow is actually quite simple to understand:

The graphics state and primitives created in the application are successively broken down in things more close to the hardware as they progress in the pipe line. One of Gallium3D's biggest achievement is defining a set of interfaces that allows the central piece -- the pipe driver --, to be reused in different graphics APIs and OSes.

If you zoom up the microscope one level, you can detect two extra (auxiliary) modules:

CSO context

Optionally used by the state tracker for Constant State Object (CSO) caching. The state passed to the pipe driver is immutable (constant) to simplify the pipe driver implementation. To avoid the performance penalty of always destroying/creating these state objects, these are stored in a cache.

Draw module

Optionally used by the pipe driver to do vertex transform, lighting, and primitive clipping in software, for hardware without support for it.

The rest...

That's all for today. Hopefully soon I'll write a bit more about these modules in more detail. I actually started my way in Gallium3D from Winsys and only recently started working on the State tracker, so there is some studying left to do.

Until then, to learn more about Gallium3D see:

• TG Wiki
• Talk from XDS 2007
• Cross-referenced source code

TextChas

I've enabled TextChas on the DRI Wiki. It is now necessary to answer a question when creating an user account and editing pages.

Two weeks have passed, and there was no new spam since TextChas were enabled, which is really nice for a change.

I'm sure spammers can try harder and beat TextChas if they are up to the challenge. But I definitely think that TextChas is a simple and effective solution for the current spam in freedesktop.org wikis. It will, at the very lest, allow us to fight spam against spammers more evenly.

Profiling DRI drivers

I've been spending a lot of time profiling DRI drivers for Gallium 3D. I've tried gprof, valgrind, and finally oprofile. Oprofile seems the best in my opinion for this purpose. More details on the DRI Wiki.

I also wrote a script to generate a time-colored call graph from oprofile output, using graphviz. See an output example of profiling glxgears on Gallium 3D:

The hotter the colour of a function is, more time is spent on that function and its children.

Using Eclipse for X.Org/DRI development

I've never learned to use Emacs. I love vim, but it doesn't scale for me when editing more than three files at the same time or projects with more than one directory. After two years using Eclipse for Python and Windows C++ development, I've been using almost exclusively Eclipse for X.Org/Mesa/DRI development during the last three months.

I think Eclipse is really a pleasant development environment:

• it has a good code navigator, which is also fast (they used Mozilla's source tree as a benchmark);
• excellent support for remote debugging (embedded companies are major contributors to Eclipse, since almost every embedded SDK nowadays is based on it);
• it rarely stands it the way (e.g., it integrates with any build system from autotools to hand written shell scripts, you still have the power of regular expression at your finger tips, etc.).

Also, the CDT (the C/C++ Development Plugin for Eclipse) folks are now actively working to get Eclipse not only to debug remote applications, but also build remotely, which will greatly simplify testing on multiple software/hardware configurations.

The thing I miss the most is better git integration. Of course, it is advisable to have a big screen and lots of memory.

For those interested, I've wrote some instructions to get you started on the X.Org Wiki.

Loading ~/.bash_profile when logging from X

In Debian/Ubuntu, when logging from gdm your ~/.bash_profile or even ~/.bash_rc are loaded, preventing you to set environment variables for all your session. The reason is that gdm starts /usr/bin/x-session-manager directly.

There are many ways to get ~/.bash_profile loaded. Many of which are wrong.

I found that a very simple way to loading ~/.bash_profile when starting X is to write the following ~/.xsession script:

#!/bin/bash --login
exec /usr/bin/x-session-manager

Mass Renaming in Nautilus

One can easily add new actions in Nautilus context menu by adding shell scripts to ~/.gnome2/nautilus-scripts . Below is a script for mass file renaming, using Perl's rename utility and zenity. Add it to ~/.gnome2/nautilus-scripts/Mass Rename.

#!/bin/sh
# Nautilus script for mass file renaming

set -e

TITLE=`basename "$0"`

# Path to perl's rename
RENAME=/usr/bin/rename


rename_all() {
        IFS=$'\n'
        for FILE in $NAUTILUS_SCRIPT_SELECTED_FILE_PATHS
        do
                cd "`dirname "$FILE"`"
                "$RENAME" "$@" "$EXPR" -- "`basename "$FILE"`"
        done
}


EXPR="s///"
while true
do
        EXPR=`zenity --title "$TITLE - expression" --entry --text "Specify the Perl expression for modifying the filenames." --entry-text "$EXPR"` || exit

        rename_all -n | sed -e 's/^\(.*\) renamed as \(.*\)$/\2\n\1/' | zenity --list --width 800 --height 600 --title  "$TITLE - Preview" --text "" --column "New name" --column "Old name" && break
done

rename_all

Here are some screenshots:

The latest version of this script can be found here.