Complicated Programming Models    

While most computers have one processor, high-performance computing often uses computers that have more than one. And as hardware prices decrease, this model is starting to become more widespread. Having more than one processor means that the threads model shown in the figure in the previous section changes to look something like this.

This figure shows four linked processors in one computer, each of which has three threads. This architecture is an extension to the model that links more than one computer together. Its advantage is that the processor doesn't need to communicate with other processors over a network, as it is completely self-contained.

The next step is to join many multiprocessor computers together. The following figure shows five computers, each with four processors, with each processsor running three threads. If this figure shows the execution of one program, then the program is using 60 threads.

This figure depicts only processors and threads. It doesn't have any information about the nature of the programs and threads or even whether the programs are copies of one another or represent different executables.

At any time, it is next to impossible to guess which threads are executing and what a thread is actually doing. To make matters worse, many multiprocessor programs begin by invoking a process such as mpirun or IBM poe, whose function is to distribute and control the work being performed. In this kind of environment, a program (or the program in a library) is using another program to control the workflow across processors.

When there are problems working this way, traditional debuggers and solutions don't work. TotalView, on the other hand, organizes this mass of executing procedures for you and lets you distinguish between threads and processes that the operating system uses from those that your program uses.