Chapter 1: Overview

Zoom is a statistical profiler and code analysis tool. Zoom periodically samples a system or a process and creates a profile from those samples. You can use Zoom to view your code and optimize its performance.

Figure 1: Zoom Profile

1.1 Requirements

1.1.1 Linux Distributions

Zoom works with Red Hat, Fedora, SuSE Linux, openSUSE, Ubuntu, or Debian with kernel 2.6.15 or later. RHEL4 with kernel 2.6.9 is also supported. Angstrom Linux and Debian Linux are supported on ARM processors. To install on other distributions, refer to the section Manual Installation in the "README" file included in the download package. Basic functionality via oprofile kernel modules is available for unsupported kernel releases.

1.1.2 Processors

Zoom uses its custom kernel modules to profile on these processors and architectures:

- Intel Atom (i386, x86-64)

- Intel Core 2, Core i7/i5/i3, Xeon (i386, x86-64)

- Intel Core, Pentium M (i386)

- Intel Pentium 4 (i386, x86-64)

- AMD Athlon (i386)

- AMD Athlon64, Athlon II (i386, x86-64)

- AMD Phenom, Phenom II (i386, x86-64)

- IBM PowerPC 970, 970FX, 970MP (ppc)

Profiling support via the oprofile kernel module is available for these processors:

- Cortex-A8, ARM 11, MPCore (arm)

- Freescale 7450, 7400, 750, e500, e300 (ppc)

- IBM Cell Broadband Engine (ppc)

1.1.3 SELinux disabled

If SELinux is set to Enforcing or Permissive, you may have to manually load and unload the kernel modules.

To disable SELinux and install the Zoom system startup scripts:

1. Edit /etc/selinux/config and set SELINUX=disabled.

2. Restart the system.

3. In a terminal run:
sudo /opt/rotateright/Zoom/setup/install_driver.sh

1.2 Components

· Zoom (the graphical user interface application) allows you to capture and review profiles. Zoom lets you navigate through a profile in a symbol-centric way. There are three top-level views of a profile: tree (top-down), leaf (bottom-up), and butterfly. In the butterfly view, selecting a symbol in the symbol list displays its call paths in the parent (bottom-up) and children (top-down) views. Double-clicking on any symbol name will open a code browser displaying its source and assembly code. The code is annotated with performance information and optimization advice.

· zoom (the command-line tool) allows you to take profiles on remote or non-GUI systems. The command-line tool may also be used in automated or scripted environments.

Chapter 2: The Zoom Workflow

Zoom performs both time and performance event sampling. In these modes, Zoom profiles the entire system – all executing processes and threads are available in the profile.

Zoom may also profile all threads in a single process – regardless of whether the threads are executing or blocked – using the "Thread Time Profile" configuration. This mode of operation may be especially useful when threads are expected to be blocked waiting on I/O or synchronization primitives such as locks.

You can create your own sampling configurations by customizing triggers and other settings. Profiles may be saved as session files that can be archived for later analysis or sharing with colleagues.

2.1 Preparation

Zoom can profile any or all running processes. Zoom can also statically analyze the code in any ELF file (the standard executable format for Linux). ELF files may be executables, object files, or library files. To correlate C, C++, or Fortran source code to machine instructions, Zoom requires debug information. This information is typically generated when the flag –g is used during compilation.

Zoom supports the debuginfo packages found in Red Hat and SUSE distributions and the dbg and dbgsym packages found in Ubuntu and Debian distributions.

2.2 Configuration

Figure 2: Built-in Sampling Configurations

There are four default sampling configurations:

Time Profile

A sample is taken on each CPU in the system every 1ms. This configuration provides a statistical view of where CPU time is being spent in the system.

Thread Time Profile

A sample is taken for each thread of a selected process every 20ms. This configuration focuses on a single process to show where its threads are executing or why they are blocked.

Cache Miss Profile

A sample is taken every 1,000 occurrences of event “Last Level Cache Misses”. This configuration exposes sections of code that are causing significant memory traffic. It may not be possible to pinpoint the exact instruction that caused a cache miss; this is a limitation of some processor implementations.

Instruction Profile

A sample is taken every 100,000 occurrences of the event “Retired Instructions”. This configuration is similar to the Time Profile, but instead of sampling based on a timer, each CPU's completed instructions are used to trigger a sample.

By default, the sample interval is dynamically increased for longer profiles to keep the total sample count manageable. This is called "Adaptive Sampling." Disabling this feature may cause sample processing time to increase significantly, but it will not generally improve the quality of the resulting profile.

You can create new sampling configurations by selecting the menu item Config->New. You can also create new sampling configurations to select custom hardware counters or events by editing the appropriate event list file for your hardware in /opt/rotateright/Zoom/share/events/. To determine which hardware event list is applicable to your system, type 'zoom events' on the command-line. Event list files are formatted in XML and should be self-explanatory. We recommend backing up the original event list file before editing it.

Figure 3: Configuration Editor

2.3 Sampling

Figure 4: Sampling Control

There are three ways to toggle sampling:

· Click on the Start/Stop button.

· Use the global Hot Key – Zoom does not need to be the foreground application. The default hot key is Control-Tab.

· Use the ZoomScript API either programmatically or with the zoomscript command-line tool.

The Target specifies the machine to be profiled. It may be the same machine that the Zoom UI is running on (localhost). The target may also be a different machine specified by hostname or IP address with a port number appended. If the target is a remote machine, command-line zoom must be running on the remote machine in “server” mode. The command-line zoom will identify the port that it will use to pass data to or from the GUI host system.

2.4 The Profile Browser

Figure 5: Profile Browser – Butterfly View

The Profile Browser allows navigation and understanding of a profile based on the code that was executing when the profile was taken. In the Leaf view, symbols are sorted based on the number of samples attributed to each symbol. Nodes may be expanded to see the callers of each symbol. In the Tree view, symbols are shown based on the hierarchy of calls in a program. For example, "main" calls "foo" which then calls "bar". In the Butterfly view, the Symbol List shows all symbols encountered in the profile for the selected process or thread. Selecting a symbol will cause its sampled call paths to be displayed in the Parents (bottom-up) and Children (top-down) views.

You can use the Process and Thread drop-down menus to focus on the time spent in a particular process or thread. You can access additional display options by right-clicking to bring up context menus. Useful shortcuts include:

· Double-clicking on the Self or Total columns will toggle the sample display mode: percent, sample count, or time.

· Control-clicking a disclosure triangle in the Leaf, Tree, Parents or Children views will automatically expand the entire subtree.

Figure 6: Filter Pane

Applications may have large call trees with thousands of symbols. Filtering allows you to consolidate sections of call trees and omit symbols that do not provide useful information. A few scenarios where filtering may come in useful are:

· Removing code that you can’t directly change – During development and testing, your code will normally include debugging information. This allows you to see source code hotspots in Zoom. Conversely, released software doesn’t usually ship with debugging information. Use Hide Symbols with No Debug Info to hide code that is not part of your development project.

· Removing calls within system libraries – Use Flatten System Modules if you don’t care about the internals of system libraries, but you would still like to know how your functions are calling the system libraries.

· Focusing on the user space portion of your application – Use Hide Supervisor Callstacks to remove kernel and driver samples and focus on your application’s user space profile.

2.5 The Code Browser

Figure 7: Code Browser

Double-click on a symbol in the Profile tab to view that symbol’s code as assembly and source code (if available).

Instructions are colored based on the number of samples – brightly colored instructions will have the most samples. You can click on the light bulbs in the first column of the code browser to see performance hints. Selecting an instruction will color its dependent instructions, allowing you to visually track code flow and potential stalls. Useful shortcuts include:

· Double-clicking on a branch instruction will take you to the branch-target if the branch-target can be resolved statically.

· Double-clicking within the Self or Total columns will toggle the sample display mode: percent, count, or time.

· Double-clicking within the Address column will toggle the address display mode: raw address or offset within a symbol.

Figure 8: Code Browser Options

The Code Browser offers several options via context menus that are available with a right-click. For example, you can navigate up or down the call tree by selecting the Open Caller or Open Callee menu items from the context menu.

2.6 Static Analysis

Figure 9: Static Analysis

Static analysis allows you to examine the contents of an ELF file for potential performance issues without actually profiling an application. Choose the menu item File->File Open… to select a file for analysis. When you open an ELF file, Zoom displays the File Summary pane, which lists all the symbols contained in the ELF file.

The File Summary pane has four tabs:

· Text — text symbols (code)

· Data — data symbols (static variables)

· Imported — imported symbols (undefined text and data)

· Labels — assembly labels in the ELF under analysis

Double-clicking a symbol in the Text table displays the Code Browser, which causes the source and assembly code to be displayed. See section 2.5 Code Browser for more information.

Double-clicking a symbol in the Data table displays the Data Browser, which shows the raw bytes for a given symbol. Alternatively, you can display the Data Browser from the Text table by right-clicking a symbol and selecting Show Data for <selected symbol> from the context menu.

Chapter 3: Advanced Profiling

There are several ways to control Zoom sampling in addition to using the GUI or command-line applications directly.

3.1 Remote Control

To profile a remote machine over a network connection, start the zoom command-line application in server mode and connect to it from the Zoom GUI.

3.1.1 Launching the Zoom Server

To launch the zoom command-line application in server mode on a remote machine:

$ zoom server

zoom is listening on port xxxx

A randomly selected communication port number is reported. You will need to specify this port number to the Zoom GUI for communication with the remote zoom server.

3.1.2 Connecting to the Zoom Server

Perform the following steps in the Sampling Control to connect to the zoom server on the remote machine.

1. Disconnect the Zoom UI from the zoom server on the local machine by clicking Disconnect.

2. Fill in Target with <host>:<port> and click Connect.

3.2 Programmatic Control (ZoomScript API)

The ZoomScript API allows you to programmatically control Zoom. To start and stop Zoom profiling from your code, include the header at /opt/rotateright/Zoom/include/zoom/zoom.h in your C/C++ code. You will also need to link your binary with /opt/rotateright/Zoom/lib/libRRZoomScript.so.

Before using the ZoomScript API, ensure that the Zoom UI is configured to allow ZoomScript control. You can do this by selecting the menu item Sampling->Allow ZoomScript Control in the Zoom UI. Alternatively, you can launch and configure the command-line zoom for programmatic control:

$ zoom run –-allow_zoomscript

Sample code for the ZoomScript API is located in /opt/rotateright/Zoom/examples/zoom/main.c.

3.2.1 API Reference

Zoom error codes are defined as an enum:

typedef enum {

ZM_OK = 0,

ZM_NO_CONNECTION = -1,

ZM_UNAVAILABLE = -2,

ZM_BUSY = -3,

ZM_ERROR = -1000,

} ZMError;

ZMError ZMConnect()

Connect to Zoom.

Return: Zoom error code enum value.

ZMError ZMDisconnect()

Disconnect from Zoom.

Return: Zoom error code enum value.

ZMError ZMStartSession()

Start sampling (must be connected).

Return: Zoom error code enum value.

ZMError ZMStopSession()

Stop sampling.

Return: Zoom error code enum value.

ZMError ZMIsSampling()

Report if Zoom is currently sampling.

Return: 1 if currently sampling, 0 otherwise.

char *ZMErrorName(ZMError error)

Report the string description for the supplied error code.

Return: A description of the error code.

3.3 Scripting (zoomscript)

zoomscript is a command-line tool that wraps the ZoomScript API and can be used in shell scripts. Before using zoomscript, ensure that the Zoom UI or zoom is configured to allow ZoomScript control.

To start profiling:

$ zoomscript start

To stop profiling:

$ zoomscript stop

The following command will start profiling, launch the executable, and stop profiling on exit:

$ zoomscript exec <executable> <args ...>

For complete usage information, execute zoomscript with no arguments.

3.4 Temporary files

During profiling, zoom writes out raw sample data and other information to the system temporary directory (the default is /var/tmp). This data can be quite large depending on the sampling interval and for how long you profile. You can specify a different path for these files by setting the ZOOM_TMPDIR environment variable. You can either set this globally in your .bash_profile (if you use the bash shell), or you can specify this directly when you launch zoom.

$ ZOOM_TMPDIR=/bigdrive zoom

Chapter 4: Troubleshooting

If you are having problems running Zoom, there are a few steps you can take to help identify the issue:

Run the command-line zoom application.

If you can capture a profile with the command-line zoom, open it in the Zoom UI.

Try profiling another machine over the network. This will help narrow down problems with the local profiling driver.

Report any bug or problems you encounter to support@rotateright.com or use the Report Bug menu item under Zoom’s Help menu. Additional information such as the version of Zoom, operating system distribution, and hardware information from /proc/cpuinfo and /proc/meminfo is often very helpful, so please include it if possible.

4.1 Profile on the Command-Line

Open a terminal window and run the following command:

$ zoom run --time_limit 1 --start immediate

If things are functioning correctly, zoom should profile for one second and quit. The file “profile_0001.zoom” should be in your current directory (the number in the file name increments with each profile taken).

If this fails, file a bug report by sending mail to support@rotateright.com including any messages or errors in the terminal as well as the log file at ~/.zoom/zoom-run.log.

4.2 Open a profile in the Zoom UI

Launch the Zoom UI from a terminal by typing Zoom and open “profile_0001.zoom” using the Open File menu item under the File menu.

If the Zoom UI fails to launch, file a bug report by sending mail to support@rotateright.com including any messages or errors reported in the terminal and any dump files generated by the Java VM.

If the Zoom UI encounters an exception, open the Help menu and file a bug report that includes the contents of the Error Log.

4.3 Profile over the network

If the Zoom UI fails to connect to the zoom daemon, there may be a problem with network communications. If any firewalls are enabled, turn them off and try to connect again. If this still fails to connect, launch the zoom server with full verbosity:

$ zoom server –-verbose 3

zoom is listening on port xxxx

In the Zoom UI, fill in Target with <host IP address>:<port> and click Connect. Do not use localhost or the loopback address for host.

File a bug report with any messages or errors reported in the terminal and the contents of the Error Log under the Help menu.

4.4 Report a bug

If possible, use the Report Bug facility under the Help menu to report any problems.

Otherwise, email your bug report to support@rotateright.com with the following information:

· Zoom version – available in the About Box from the Help menu

· Linux distribution and kernel version – the output of uname -a

· Platform (e.g. x86_64, i386, ppc) – the output of arch

· Processor (e.g. 4 x 2.40GHz Intel Core 2) – the contents of /proc/cpuinfo

· Amount of system memory (RAM) – the contents of /proc/meminfo